Identifying conflict of interest disclosure in systematic reviews of surgical interventions and devices: a cross-sectional survey

doi:10.21203/rs.3.rs-33056/v1

Download PDF

Research article

Identifying conflict of interest disclosure in systematic reviews of surgical interventions and devices: a cross-sectional survey

https://doi.org/10.21203/rs.3.rs-33056/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 19 Oct, 2020

Read the published version in BMC Medical Research Methodology →

You are reading this older preprint version

Read the latest preprint version →

Background

Conflict of interest (COI) is an important source of bias in research and disclosure is the most frequent strategy to manage COI. Considering the importance of systematic reviews (SRs) and the prevalence of COI is varying among different research fields and the, we conducted a survey to identify the range of conflict of interest in SRs assessing surgical intervention and devices, and explored the association between COI disclosure and conclusion.

Methods

We retrieved SRs of surgical interventions and devices published in 2017 via PubMed. Information regarding general characteristics, funding source and COI disclosure were extracted. We conducted a descriptive analyses of study characteristics of included systematic reviews. The difference between groups in the authors’ conclusions were compared using the Chi-square test. Results were expressed as odds ratio and its 95% confidence interval.

Results

155 SRs were finally identified in 2017, more than half of SRs (58.7%) reported their funding source and 94.2% reported authors’ COI disclosure. Among 146 SRs that stated COI disclosures, only 35 (22.6%) SRs declared at least one author had a COI. More than 40 terms were used to describe COI. Cochrane SRs were more likely to provide a detail description of COI comparing with those in non-CSR (48.0% versus 25.4%, P = 0,023). No association between positive conclusion and COI disclosure was found (P = 0.484, OR = 0.43, 95%CI: 0.08, 2.16). In the subgroup analyses, SRs stating no COI disclosure were more likely to report positive conclusion than those stating at least one type of COI, but the difference is not significantly different (P = 0.406, OR = 1.38, 95%CI: 0.64, 2.98)

Conclusion

A high rate of COI disclosure without any detailed information. Although little impact of COI disclosure on author’ conclusions in SRs was found for limited sample, clear description of all potential COI is the best way to enhance the credibility of published scholarship and unbiased evidence-informed decision.

Health Economics & Outcomes Research

Conflicts of interest

Systematic reviews

Surgical interventions

devices

The interactions between surgeons and industry are a benefit for the surgical innovation, but the interest also could undermine public trusts in surgeons [1]. Conflict of interest (COI) is defined as “a set of circumstances that creates a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest” [2] and considered to be “ubiquitous and inevitable in academic life” [3, 4]. Full disclosure is acknowledged as an important method for reporting and managing conflict of interest and serves to highlight the potential for bias [5–8]. Since 1988, International Committee of Medical Journal Editors (ICMJE) has required authors to disclose financial or other relationships that might lead to a conflict of interests [9], and hundreds of biomedical journal then stated that they followed the policy of ICMJE [10].

The number of studies regarding COI increased considerably in recently year because of the concerns about competing interests between researchers and manufactures [11–19]. In 2017, the Lancet published a collection to discuss issues relating to conflict of interests. Transparent disclosure of conflict of interests is increasingly emphasized as an important component in the reporting of both clinical trials and systematic review after then [17–19]. Although a lot of studies addressed the conflict of interest in medical research were published, a limited number of studies assessing COI disclosure, especially on non-financial interest in surgical intervention and devices was found for the disproportionate published number between medical and surgical studies [20–22].

Considering the importance of SRs on clinical guideline development and policymaking and the prevalence of COI is varying among different research fields [23], we, therefore, performed a survey to identify the range of COI in systematic reviews assessing surgical intervention and devices, and tried to provide a potential checklist for COI disclosure about surgical research. We also explored the impact of COI disclosure on study conclusion.

Eligibility criteria

A study was included if (1) described as a systematic review or a Meta-analysis; (2) included a search strategy of at least one database; (3) only included RCTs assessing at least one surgical intervention. Network Meta-analyses, methodological systematic reviews or systematic reviews reported as conference abstracts and research letters were excluded. A systematic review was defined as described in the Cochrane handbook (version 6, 2019) [24] and the definitions of surgical intervention and device have been described elsewhere [25]. A COI disclosure was defined as whether a COI disclosure was stated or not.

Data resource and study procedures

Systematic reviews published in 2017 were identified through PubMed. The search strategy was based on MeSH terms and their variants regarding SR and surgical intervention, was developed in collaboration with an experienced librarian (Additional file 1). Two reviewers (JY and GS), trained in trial and systematic review methods, used predefined, pilot-tested forms independently to screen abstracts and full texts for eligibility. Disagreement was resolved by consensus.

Data collection

Two teams of reviewers independently collected data from eligible studies. Discrepancies were resolved by discussion. Coher’sκstatistics was used to assess agreement between reviewers.

The following characteristics information was collected for each eligible study: (1) number of authors; (2) country of corresponding author; (3) number of trials included; (4) total number of participants involved; (5) review type (Cochrane SR or non-Cochrane SR); (6) adherence to PRISMA (7) type of journal (general or surgical journal); (8) type of control (surgical intervention, non-surgical intervention or both); (9) involvement of methodologist (i.e. statistician, epidemiologist). We judged that a systematic review involved a methodologist if any of the authors were affiliated with a department of epidemiology, statistics, or evidence-based medicine, or if a methodologist was listed in the acknowledgment section.

We extracted the funding source and classified them as industry funding (e.g. funding from device industry; company; insurance company), non-industry funding (e.g. funding from academic institution; university; hospital; government bodies; foundation; charity, et al), no funding and not reported. We extracted the statement of COI verbatim and then categorized them based on their characteristics and nature for avoiding the information missing.

Authors’ conclusion was labelled “positive” when the overall results favored intervention over control or both groups were equal but the interventional group had minor advantages. A “negative” conclusion was defined if the results favored control or both groups were equal [26].

Data analysis

We conducted a descriptive analysis of study characteristics of included systematic reviews. For all descriptive analyses, frequencies (and percentages) were used for dichotomous variables, and mean (and standard deviation) or median (and range) for continuous variables. We compared funding source and COI disclosure between Cochrane SR and non-CSR using the Fisher’s exact test for dichotomous variables.

We categorized the studies into 2 groups based on whether the investigators disclosed COI or not. If they did, we then divided the COI disclosure group into 2 subgroups (at least one type of COI was disclosed and no COI disclosure group). The difference between groups in the authors’ conclusions were compared using the Chi-square test. Results were expressed as odds ratio and its 95% confidence interval. All analyses were performed by IBM SPSS Statistics 26.

A total of 6256 studies were identified from PubMed. After the title and abstract screening, 479 were found to be potentially eligible, and 155 systematic reviews were finally included based on the full-text articles (Figure). The interobserver agreement on data collection was good (κ = 0.84). A full list of the included SRs was presented in additional file 2.

General information from the included SRs

The general characteristics are listed in Table 1. The median number of included trials was 6 (range 2 to 21), the median number of participants and databases included were 1167 (range 80 to 19886) and 4 (range 1 to 15), respectively. 25 (16.1%) were Cochrane SRs and 89 (57.4%) mentioned PRISMA in their articles. 109 (70.3%) were published in general medical journals; 23 (16.1%) involved researchers with affiliations with either epidemiology or statistics departments; 132 (85.2%) tested alternative surgical interventions and 79 (59.8%) evaluated the efficacy or harms on different procedures.

Table 1

General characteristics of included systematic review
Characteristics	n = 155
No of authors ^a	6 (2–21)
No of study included ^a	7 (1–44)
No of participants included ^a	1167 (80-19886)
No of database included ^a	4 (1–15)
Methodologist involved	23 (14.8)
Cochrane SR	25 (16.1)
PRISMA mentioned	89 (57.4)
Type of journal
General	109 (70.3)
Surgical	46 (29.7)
Country of corresponded author
China	48 (31.0)
USA	22 (14.2)
UK	18 (11.6)
Italy	12 (7.7)
Australia	12 (7.7)
Other	43 (27.8)
Type of comparison
Surgical versus surgical	132 (85.2)
Surgical versus non-surgical	17 (10.9)
Surgical versus both	6 (3.9)
Comparisons between different surgical procedures
Procedures	79 (59.8)
Devices	53 (40.2)
Specialty
General	54 (34.9)
Cardiothoracic	45 (29.0)
Orthopedic	24 (15.5)
Other	32 (20.6)
Values in parentheses are percentages unless indicated otherwise
^a values are median (range)

Funding information from the included SRs

Of the 155 selected SRs, 42 (27.1%) reported review funding from non-industry sources, mostly from governmental agencies and academic institutions. Three (1.9%) reported review funding from pharmaceutical or devices company, 45 (29.0%) stated the review was not funded; and still 64 (41.3%) did not provide review funding information. Compared with non-Cochrane SRs, Cochrane SRs (CSR) were more likely to report the information of funding source, including those without funding (53.8% versus 84%, P = 0.005)

COI disclosure concerning the included SRs

One hundred and forty-six studies (94.2%) disclosed the information of authors’ COI. However, of the 146 systematic reviews that provided COI disclosures, only 35 (22.6%) SRs declared at least one author had one COI. More than 40 terms were used to describe COI and we categorized them into 12 terms that were the most common descriptions of COI disclosure, including grants/fellowship, honoraria, consulting, employment/salary, patent/copyright, equity/stocks/bonds, non-monetary support, service in other affiliations, founder or other leadership in a company, intellectual beliefs, experience and personal relationship.

Of the 35 SRs that stated COI, Cochrane SRs were more likely to provide a detail description of COI comparing with those in non-CSR (48.0% versus 25.4%, P = 0,023). The most frequently reported type was “grant/fellowship” (n = 17, 11.0%), followed by “honoraria” (n = 15, 9.7%), “consulting” (n = 14, 9.0%), “non-monetary support” and “services in other affiliations” (n = 8, 5.2%) (Table 2).

Table 2

Classification of funding source and COI in the included systematic reviews
	Overall (n = 155)	CSR (n = 25)	Non-CSR (n = 130)
Source of funding
Industry funding	3 (1.9)	0 (0)	3 (2.3)
Non-industry funding	42 (27.1)	20 (80.0)	22 (16.9)
Industry + non-industry funding	1 (0.6)	0 (0)	1 (0.8)
Reported as not funded	45 (29.0)	1 (4.0)	44 (33.8)
Not reported	64 (41.3)	4 (16.0)	60 (46.2)
Types of COI
No COI to disclose	111 (71.6)	12 (48.0)	99 (76.2)
At least one type
Grant/fellowship	17 (11.0)	6 (24.0)	11 (8.5)
Honoraria	15 (9.7)	2 (8.0)	13 (1.0)
Consulting	14 (9.0)	4 (16.0)	10 (7.7)
Non-monetary support	8 (5.2)	5 (20.0)	3 (2.3)
Service in other affiliations	8 (5.2)	2 (8.0)	6 (4.6)
Equity/stocks/bonds	3 (1.9)	1 (4.0)	2 (1.5)
Patent	2 (1.3)	0 (0)	2 (1.5)
Employment/salary	1 (0.6)	1 (4.0)	0 (0)
Leadership in company	1 (0.6)	0 (0)	1 (0.8)
Intellectual beliefs	6 (3.8)	4 (16.0)	2 (1.5)
Experience	3 (1.9)	2 (8.0)	1 (0.8)
Personal relationship	2 (1.3)	2 (8.0)	0 (0)
Values in parentheses are percentages
CSR: Cochrane systematic review

We found no significant difference between SRs had COI disclosure and those did not (P = 0.484, OR = 0.43, 95%CI: 0.08, 2.16). In the subgroup analyses, SRs stating no COI disclosure were more likely to report positive conclusion than those stating at least one type of COI, but the difference is still not significantly different (P = 0.406, OR = 1.38, 95%CI: 0.64, 2.98).

94.2% of surgical systematic reviews published in 2017 reported authors’ COI disclosures in our survey. This result probably caused by that more than 95% of medical journals had a COI policy taking disclosure as an essential part of biomedical studies [16]. However, among 147 studies declared their COI, only 35 (22.6%) declared at least one of COI in their study.

There are several possible reasons for the low rate of detailed COI disclosure. One of the reasons for this is that few researchers are unaware some behaviors including receiving support for food and beverages, academic competition are also conflicts of interests and could also affect their research [9, 27]. Another potential possibility is the policies regarding some COI are constantly obscure, such as non-financial COI [16, 28–30]. We reviewed “instruction to authors” on journals’ website which publishing our including SRs, and most of the COI policies only involve some specific types of COI, such as grant, consulting fee, employment and so on. For help researchers to identify their potential COI, we provided a checklist of COI that enable authors to improve the accuracy and completeness of their COI disclosure, and the journals also could consider to include in their policies for authors’ disclosure of COI (Table 3).

Table 3

A potential checklist of conflicts of interest disclosure
Proposed terms	Descriptions
Financial COI
Personal fees/payment	Fees paid to person for consulting, lecturer, speakers bureaus, expert testimony, presentations, manuscript preparation, educational support, writing and reviewing assistant
Employment/salary	The professional is/was employed by a company and a periodic payment is/was provided by the company
Non-monetary support	Travel paid, accommodations, meeting expenses, administrative support; food; beverage
Drug/equipment supplies	The provision of surgical or research devices by a company
Grant	Grant from a company, or governmental agency, hospital, university or other institutions
Patent(s)/copyright	The professional holds or shares a patent
Equity/stocks/bonds	Ownership of interest in a company
Non-financial COI
Service in other affiliates	Such as scientific advisory board, steering committee membership
Leadership in a company	The professional holds the position of a high ranking corporate officer in a company and has responsibility for its operation
Intellectual beliefs	Authorship of primary studies included or not include in the SR; Participation in a previous guideline panel or editorial
Faith and fixed beliefs	Such as political beliefs, religious beliefs, culture practices and dietary habits
Personal experience	Specialty training; experience with specific population; personal preference
Personal relationship	Social relationship
Academic competition	Driven by some competitive academic or desire for glory

Several methodological studies have been conducted to assess the reporting of COI disclosure, and there is a substantial variability in the reporting rate of COI among different study designs and specialties (17.0–71.2%) [16–19, 21, 22, 33–36]. Generally, RCT and cohort studies preferred to provide more detailed information on COI than those in systematic reviews. Our finding on the rate of COI disclosure is largely consistent with those studies assessing SRs [16–18] and no significant difference on rates of COI was found between trials in drug and surgical field.

Conflict of interest in biomedical research has been associated with positive conclusion and suppression of negative results [19, 20, 37–39]. However, we did not find the association between positive conclusion and COI disclosure in our study. Considering Cochrane’ policy regarding funding and COI are stricter than other journals [27], we also conducted a sensitivity analyses that only included non-CSR and the result was still the same. The interpretation of this finding should be caution for a lower rate of including SRs in our study were sponsored by the industry comparing with previous studies [19, 37].

The present study included a large sample of SRs with a wide range of surgical interventions, which should lead to better generalizability of findings. We used systematic methods for data abstraction and provided a potential checklist of COI to readers and editors. However, the study also has some limitations. First, these disclosures have all been reported by authors, and we have limited method to confirm or verify them. Second, we only assessed those SRs including randomized controlled trial, and it could have underestimated the rate of disclosure in SRs stated at least one type of COI.

In summary, 94% of SRs stated their COI disclosure and a high rate of COI disclosure without detailed information. Although little impact of COI disclosure on author’ conclusions in SRs was found for limited sample size, clear description of all potential COI disclosure is still the best way to enhance the credibility of published scholarship, help researchers and policymakers in building unbiased evidence-informed decision. The relationships between surgeons and industry or society are more complicated than those we referred to in this study, further explorations should be continued to explore the related activities.

Ethical approval and consent to participate: Not applicable

Consent for publication: Not applicable

Availability of data and materials: The dataset supporting the conclusions of this article is included within the article and its additional files.

Conflict of interests: All authors declare no conflict of interest

Funding: This study was supported by a grant from “National Natural Science Foundation of China” (project No: 71704120), “National Science and Technology Major Projects” (project No: 2018ZX09201009-005-003) and International visiting program for Excellent Young Scholars of SCU

Authors' contributions: JY conceived and designed this study, searched the literature, extracted data, synthesized data, and developed the first draft of the manuscript; GS, ZY and YZ carefully checked studies and extracted data; HA provided critical methodological guidance. All authors critically revised the manuscript.

Keune JD, Vig S, Hall BL, Matthews BD, Klingensmith ME. Taking disclosure seriously: disclosing financial conflicts of interest at the American College of Surgeons. J Am Coll Surg. 2011;212(2):215–24.
Field MJ, Lo B. Conflict of Interest in Medical Research, Education, and Practice. Washington, DC: National Academies Press; 2009.
DeAngelis CD, Fontanarosa PB, Flanagin A. Reporting financial conflicts of interest and relationships between investigators and research sponsors. JAMA. 2001;286(1):89–91.
Fontanarosa PB, Flanagin A, DeAngelis CD. Reporting conflicts of interest, financial aspects of research, and role of sponsors in funded studies. JAMA. 2005;294(1):110–1.
Jureidini JN, McHenry LB. Conflicted medical journals and the failure of trust. Account Res. 2011;18(1):45–54.
Eliades T, Turpin DL. Conflict of interest: always report it, and if in doubt, ask. Am J Orthod Dentofacial Orthop. 2008;134(3):327–8.
Cooper RJ, Gupta M, Wilkes MS, Hoffman JR. Conflict of Interest Disclosure Policies and Practices in Peer-reviewed Biomedical Journals. J Gen Intern Med. 2006;21(12):1248–52.
Lipton S, Boyd EA, Bero LA. Conflicts of interest in academic research: policies, processes, and attitudes. Account Res. 2004;11(2):119–39.
International Committee of Medical Journal editor. Uniform requirements for manuscripts submitted to biomedical journals. BMJ. 1991;302(6772):338–41.
Drazen JM, Van der Weyden MB, Sahni P, et al. Uniform format for disclosure of competing interests in ICMJE journals. N Engl J Med. 2009;122(1305):12–4.
Moraes FY, Leite ETT, Hamstra DA, et al. Self-reported Conflicts of Interest and Trial Sponsorship of Clinical Trials in Prostate Cancer Involving Radiotherapy. Am J Clin Oncol. 2018;41(1):6–12.
Edwards IR. Conflicts of interest in medicines safety and regulation: how much conflict and how much interest should we allow? Am J Manag Care. 2011;17(4):e148.
Ngo-Metzger Q, Moyer V, Grossman D, et al. Conflicts of Interest in Clinical Guidelines: Update of U.S. Preventive Services Task Force Policies and Procedures. Am J Prev Med. 2018;54(1S1):70–80.
Klanica K. Conflicts of interest in medical research: how much conflict should exceed legal boundaries? Ann Intern Med. 2006;144(3):225.
Kaestner V, Prasad V. Financial conflicts of interest among editorialists in high-impact journals. Blood Cancer J. 2017;7(9):e611.
Hakoum MB, Anouti S, Al-Gibbawi M, et al. Reporting of financial and non-financial conflicts of interest by authors of systematic reviews: a methodological survey. BMJ Open. 2016;6(8):e011997.
Roseman M, Turner EH, Lexchin J, Coyne JC, Bero LA, Thombs BD. Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. BMJ. 2012;345:e5155.
Roseman M, Milette K, Bero LA, et al. Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments. JAMA. 2011;305(10):1008–17.
Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454–65.
Okike K, Kocher MS, Mehlman CT, Bhandari M. Conflict of interest in orthopaedic research. An association between findings and funding in scientific presentations. J Bone Joint Surg Am. 2007;89(3):608–13.
Agha RA, Barai I, Rajmohan S, et al. Support for reporting guidelines in surgical journals needs improvement: A systematic review. Int J Surg. 2017;45:14–7.
Probst P, Huttner FJ, Klaiber U, Diener MK, Buchler MW, Knebel P. Thirty years of disclosure of conflict of interest in surgery journals. Surgery. 2015;157(4):627–33.
Adie S, Ma D, Harris IA, Naylor JM, Craig JC. Quality of conduct and reporting of meta-analyses of surgical interventions. Ann Surg. 2015;261(4):685–94.
Cochrane Handbook for Systematic Reviews of Interventions. (version 6, 2019). https://training.cochrane.org/handbook/current. Accessed 15 Aug 2019.
Yu J, Li X, Li Y, Sun X. Quality of reporting in surgical randomized clinical trials. Br J Surg. 2017;104(3):296–303.
Kjaergard LL, Als-Nielsen B. Association between competing interests and authors' conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ. 2002;325(7358):249.
Cochrane Community. Disclosure of Potential Conflicts of Interest form: Instructions https://community.cochrane.org/help/tools-and-software/archie/resources-archie/disclosure-potential-conflicts-interest-form-instructions. Accessed 15 Jan 2020.
Grundy Q, Mayes C, Holloway K, Mazzarello S, Thombs BD, Bero L. Conflict of interest as ethical shorthand: understanding the range and nature of "non-financial conflict of interest" in biomedicine. J Clin Epidemiol. 2019;120:1–7.
Hakoum MB, Jouni N, Abou-Jaoude EA, et al. Authors of clinical trials reported individual and financial conflicts of interest more frequently than institutional and nonfinancial ones: a methodological survey. J Clin Epidemiol. 2017;87:78–86.
Shawwa K, Kallas R, Koujanian S, et al. Requirements of clinical journals for authors? Disclosure of financial and non-financial conflicts of interest: A cross sectional study. PLoS ONE. 2016;11(3):e0152301.
Wiersma M, Kerridge I, Lipworth W. Dangers of neglecting non-financial conflicts of interest in health and medicine. J Med Ethics. 2018;44(5):319–22.
Wiersma M, Kerridge I, Lipworth W, Rodwin M. Should we try to manage nonfinancial interests? BMJ. 2018;361.
Lerner TG, Miranda Mda C, Lera AT, et al. The prevalence and influence of self-reported conflicts of interest by editorial authors of phase III cancer trials. Ann Intern Med. 2012;156(11):809–16.
Hakoum MB, Jouni N, Abou-Jaoude EA, et al. Characteristics of funding of clinical trials: cross-sectional survey and proposed guidance. BMJ open. 2017;7(10):e015997.
Cherla DV, Olavarria OA, Bernardi K, et al. Investigation of Financial Conflict of Interest among Published Ventral Hernia Research. J Am Coll Surg. 2018;226(3):230–4.
Bridoux V, Moutel G, Schwarz L, Michot F, Herve C, Tuech JJ. Disclosure of funding sources and conflicts of interest in phase III surgical trials: survey of ten general surgery journals. World J Surg. 2014;38(10):2487–93.
Lundh A, Barbateskovic M, Hrobjartsson A, Gotzsche PC. Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue - cohort study. PLoS Med. 2010;7(10):e1000354.
Cherla DV, Viso CP, Olavarria OA, et al. The Impact of Financial Conflict of Interest on Surgical Research: An Observational Study of Published Manuscripts. World J Surg. 2018;42(9):2757–62.
Fabbri A, Gregoraci G, Tedesco D, et al. Conflict of interest between professional medical societies and industry: a cross-sectional study of Italian medical societies' websites. BMJ open. 2016;6(6):e011124.

Download PDF

Journal Publication

published 19 Oct, 2020

Read the published version in BMC Medical Research Methodology →

Editorial decision: Major revision
26 Aug, 2020
Review #2 received at journal
24 Aug, 2020
Review #1 received at journal
17 Aug, 2020
Reviewer #2 agreed at journal
10 Aug, 2020
Reviewer #1 agreed at journal
28 Jul, 2020
Reviewers invited by journal
25 Jul, 2020
First submitted to journal
01 Jun, 2020
Submission checks completed at journal
01 Jun, 2020
Editor invited by journal
01 Jun, 2020
Editor assigned by journal
01 Jun, 2020

You are reading this older preprint version

Read the latest preprint version →

Identifying conflict of interest disclosure in systematic reviews of surgical interventions and devices: a cross-sectional survey

Status:

Journal Publication

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

Introduction

Methods

Results

Discussion

Conclusions

Declarations

References

Supplementary Files

Status:

Journal Publication

Version 1