Prior to the beginning of the review, a protocol was produced, which included the search strategy, inclusion criteria, and outcomes of interest. The protocol has been registered on the International prospective register of systematic reviews (CRD42018094628). Comparing with the protocol, the changes in the process of the review conduction were listed in Additional file 1.
Patient and Public involvement
All the data of our study were extracted from published articles and contained no study with human participants or animals performed by any of the authors. Informed consent was not involved since all the data in this study were extracted from published articles. Meanwhile, no patient involved in the study, hence, there was no need to make dissemination.
Studies Selection
We searched in Medline/OVID from 1946 to March 1st, 2019, EMBASE/OVID from 1974 to March 1st, 2017, Cochrane Database of Systematic Reviews (CDSR), Cochrane Central Register of Controlled Trials, International prospective register of systematic reviews (PROSPERO), Psyc ARTICLES/OVID, Chinese Bio-Medical Literature Database (CBM), China National Knowledge Infrastructure (CNKI), Wan Fang Data and VIP Database on March 1st, 2019 to find relevant systematic reviews or meta-analysis. The specific search strategies of the above electronic searches are listed in Additional file 2. We also searched the bibliographies of the included studies.
Inclusion criteria (the included systematic reviews or meta-analyses should meet all the following criteria): (1) appraise the long-term effectiveness or efficacy of opioids for CNCP;(2) Have long-term results–identified as lasting more than 4 weeks; (3) opioids should be compared with a placebo or non-opioid analgesics;(4) state explicitly that the study was a systematic review or meta-analysis; (5) be published in English or Chinese.
Exclusion criteria (systematic reviews or meta-analyses that met any of the following criteria would be excluded): (1) includes an appraisal of chronic cancer pain; (2) had a period of less than 4 weeks; (3) the study was not a systematic review or meta-analysis; (4) the systematic review or meta-analysis included the study of animal trials.
Studies selection
Two reviewers (Qian Li and Xiaoyuan Jiang) undertook the process of study selection independently according to the inclusion and exclusion criteria, and any controversial systematic reviews or meta-analyses were resolved by discussion. If necessary, the third reviewer (Huan Tao) was involved in judgment. The specific steps were listed as follows: Firstly, the two review authors downloaded all relative literature after the search was complete. Then they imported the literature to an Endnote database (Version 7.0). Next, they screened the title and abstract of all identified studies according to the inclusion and exclusion criteria. Finally, they retrieved all possibly relevant articles in full text to assess their internal validity (quality) and whether they satisfy the inclusion criteria.
Data extraction and management
Two reviewers (Qian Li and Hui Liu) independently extracted relevant data to a data collection template according to the AMSTAR-2 questionnaire. The third reviewer (Ke Deng) was consulted when disagreements were not resolved by discussion. The specific items extracted from the full text are as follows: journal name, first author’s name, number of authors, number of authors with a conflict of interest, country, year of publication, published journal, impact factor of the published journal, searching database, number of searching databases, additional retrieval, whether the review was registered, whether the protocol was published, number of included studies, type of included study RCT or NRSI), drug, treatment duration, instrument to evaluate the risk of bias, whether go on meta-analysis, primary outcome, and conclusion.
Methodological quality assessment by the tool of AMSTAR-2
Two reviewers (Qian Li and Xiaoyuan Jiang) were trained to assess the methodological quality of systematic reviews or meta-analyses by an experienced reviewer (Jin Chen). They independently conducted the process of methodological assessment. Each reviewer provided reasons for their judgment and disagreements were solved by face-to-face discussion. The appraisal tool of AMSTAR-2 included 16 domains: whether there was PICO element in the research issues and the inclusion criteria, the protocol of systematic review or meta-analysis, the reason for selecting the study design, the literature searching strategy, study selection, data extraction, specific details of inclusion and exclusion criteria, adequate detail of the included studies, bias risk assessment of the included studies, the sources of funding, appropriate statistical methods, the impact evaluation of the individual study’s RoB (risk of bias), the explanation of RoB in individual studies, a satisfactory explanation for any heterogeneity, adequate investigation of publication bias, and potential conflicts of interest.
The answer options of the AMSTAR-2 were Yes, Partial Yes, and No. “Yes” denoted a positive result. “No” represented that there was not enough information about the domain. “Partial Yes” represented that it partially adhered to the standard. AMSTAR-2 used BOX 1 of the seven critical domains — item2, item4, item7, item9, item11, item13, item15 — as the critical domains, while the other nine domains were non-critical. It then used BOX 2 to rate overall confidence of the reviews, including four ranks: High (no or one non-critical domain weakness), Moderate (more than one domain weakness but no critical domain weaknesses), Low (one critical flaw with or without non-critical weaknesses), and Critically Low (more than one critical flaw with or without non-critical weaknesses).
Statistical analysis
Kappa (κ) was used to measure the inter-rater reliability (IRR) between the two reviewers for AMSTAR-2. κ value ≤ 0 indicated no agreement, 0.01–0.20 slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement, and 0.81–1.00 almost perfect agreement21. Data were summarized as frequencies or percentages for categorical variables and as mean ± standard deviation or median (interquartile range: the 25th to 75th percentile) for continuous variables, such as the import factors of the published journals. A non-parametric test (Chi-square test) was used to assess whether there was any difference between Cochrane reviews and Non-Cochrane reviews in each domain. The association between the number of authors, the number of searched databases, the import factors of the published journal, the number of authors with a conflict of interest, and the AMSTAR-2 overall confidence for each study was analyzed by Spearman correlation analysis. The association between the year of publication, the continent of publication, the type of included studies, whether it is a Cochrane review or not, the source of funding, and the AMSTAR-2 overall confidence was analyzed by non-parametric tests (rank-sum test). We used SPSS 20.0 for statistical analysis, and statistical significance was identified as two-sided, P <0.05.