Eligibility criteria
We included a study if it was an RCT, assessed the effects of acupuncture (i.e., manual or electrical needle stimulation) versus any type of control treatment, examined efficacy/effectiveness with or without safety outcomes, and was a main study report. We excluded trial protocols, pilot trials, or RCTs reported as a research letter.
Literature search
We searched PubMed to identify reports of acupuncture RCTs published in the core clinical journals as well as in complementary and alternative medicine (CAM) journals from January 2010 to December 2019 (10 years). The core clinical journals, previously known as the Abridged Index Medicus, were defined by the US National Library of Medicine, and included 118 journals published in English in 2020, covering all specialties of clinical medicine and public health sciences [22]. CAM journals were defined by the SCImago Journal & Country Rank (SJR) website, and comprised 55 journals indexed in Web of Science in 2019 [23]. We used MeSH and free texts terms related to acupuncture, RCT and journals to search for potentially eligible studies, and one experienced investigator (JL) developed the search strategy with reference to a previous related study [24] (Appendix 1).
Study process
Two paired methods-trained researchers, who had experiences in acupuncture trials, used pilot-tested, standardized forms, together with detailed written instructions to screen abstracts and full texts for eligibility, and to abstract data from eligible studies independently and in duplicate. Disagreements were resolved through discussion or, if needed, adjudication by a third researcher (JL or LL).
Data collection
To assess implementation of interventions and selection of control among acupuncture RCTs, we developed a questionnaire to collect data from included trials. The original version was developed by three experienced authors (JL, LL and XS) with reference to revised STRICTA [17] and previous methodological studies [25-27]. Then our study group piloted the questionnaire, and convened to discuss the appropriateness and applicability of included items. After further revising the questionnaire, we conducted the second pilot and addressed discrepancies through group discussion. Based on the piloted questionnaire, we collected the following information from eligible trials:
1) General characteristics: year of publication, journal type, hypothesis of the trial, type of design, international trial, center involved, country of trial conducted, sample size, length of follow-up, number of authors, methodologists involved, source of funding, study setting, participants recruitment, trial registered, protocol provided, exclusion of patients with acupuncture experience, and Traditional Chinese Medicine (TCM) diagnosis reported.
We judged that a trial involved a methodologist if any author declared an affiliation with a department of epidemiology, statistics, or evidence-based medicine, or if a methodologist was listed in the acknowledgement section.
2) Acupuncture rationale:style of acupuncture specified, style of acupuncture, acupoint prescription, reasoning for acupuncture treatment, rationale for acupoint selection, and standards of acupoint positioning.
3) Acupuncture details: we collected details of needling, treatment regimen, other components of treatment, and practitioner background from eligible trials. Details of needling included needle stimulation, number of needle insertions per subject per session, names of points used, response sought, etc. Treatment regimen included total number of treatment sessions, actual number of treatment sessions specified, incomplete treatment sessions specified, frequency of treatment sessions, duration of treatment sessions, etc. Other components of treatment includeddetails of other interventions administered to the acupuncture group, and setting and context of treatment (i.e., instructions to practitioners, and information and explanations to patients). Practitioner background included description of participating acupuncturists (qualification or professional affiliation, years in acupuncture practice, other relevant experience).
4) Control selection and rationale:type of control, rationale for the control selection, precise description of the control, and details of sham acupuncture (e.g., sham acupoints, shallow needling, blunt needle device).
5) Main effect of pain outcome in trials of acupuncture for chronic pain: name of pain outcome, instrument used to measure the pain intensity, mean and standard deviation (SD) of changes form baseline in pain intensity, and number of patients included for analyses in each treatment group. If mean and SD of changes form baseline of pain intensity were not available, we collected mean and SD at baseline and last follow up.
One pain outcome was selected in each trial of acupuncture for chronic pain. If the primary outcome of a trial was pain intensity, we selected it as the primary pain outcome for our analyses; Otherwise, we selected the first reported pain intensity outcome from secondary outcomes for our analyses. If a trial did not clearly specify the primary or secondary outcomes, we selected one outcome according to our previously published strategy [27, 28].
Statistical analysis
We conducted descriptive analyses of study characteristics, rationale and details of acupuncture interventions and control selection of included trials. For all descriptive analyses, we used frequencies (and percentages) for dichotomous variables, and mean (and SD) or median (and range) for continuous variables. We used Chi-Square test or Fisher's exact test to compare characteristics, interventions implementation, and control selection of trials published in core clinical journals and CAM journals for dichotomous variables, and t-test for continuous variables when the distribution was normal or Mann Whitney U test otherwise. We used R software (version 3.6.3) for data analysis, and a P value of 0.05 or less than 0.05 were considered statistically significant.
As most included trial assessing the effects of acupuncture for chronic pain, we conducted network meta-analyses to investigate whether there was a differential effect between sham acupuncture and waiting-list/no treatment groups in patients with chronic pain. We also conducted two separate analyses by type of sham acupuncture (i.e., non-penetrating vs. penetrating) to explore whether non-penetrating or penetrating sham acupuncture had differential effect. The primary outcome was changes form baseline of pain intensity measured by scales such as 10-cm Visual Analog Scale (VAS), and an 11-cm point numeric rating scale (NRS). For trials that did not report changes form baseline in pain intensity, we calculated the mean change from baseline and imputed its SD according to approaches described in Cochrane handbook[29]. We conducted frequentist pairwise, indirect and network meta-analyses using a random-effects model by inverse variance method, with standardized mean difference (SMD) and corresponding 95% confidence intervals. We used node splitting method to check for consistency of direct and indirect evidence in our network.