This study is a secondary analysis of the clinical trials included in Cochrane oral health systematic reviews. Cochrane review authors have made all the judgements concerning the quality of the RCTs.
Data extraction and management
This phase of the study was carried out for all the included trials by seven independent authors (AS-M, PI, MS, ES, HM, SN, PM), and the results were entered directly into preformatted Excel 2016 spreadsheets. In order to calibrate the authors, they were all trained regarding the protocol by extracting the information from 50 random RCTs as a pilot. All the spreadsheets were then merged into one file and rechecked to avoid any mistakes and inconsistencies among the data extractors (AS-M, PI, MS). The items intended to be extracted were divided into three categories: (1) general information of Cochrane reviews including the type of review, the subject, the number of included trials; (2) basic information on the RCTs, e.g. title, first author, year of publication, journal, etc.; (3) the information reported in the RCTs (the “Characteristics of the included studies” section), which characterised the main methodological features of each RCT; and (4) the RoB assessment of the RCTs. Additionally, using the first authors’ affiliations, we assessed the distribution of the RCTs’ locations. We also evaluated the Cochrane reviews’ subjects (defined by Cochrane Library’s classification for dentistry and oral health) with the highest citations according to the Web of Science. The mean number of citations per subject was measured, as well.
The following characteristics in the journals with more than three RCTs in the Cochrane reviews were assessed as well: their ranking, subject (according to SCImago Journal Rank 2019), and the number of inclusions in the Cochrane reviews.
Given that the Cochrane review authors evaluated the blinding process differently, the RoB assessments were performed by different trajectories. In some studies, blinding took place in two subgroups, i.e. blinding of the participants and personnel (performance bias) and blinding of the outcome assessment (detection bias); meanwhile, in some other studies, blinding was taken as one item. In rare instances, blinding took place in two categories, including subjective and objective outcomes or a range, namely, assessor, analyst, participants, and caregivers. In these cases, an aggregated form was provided; if one or more subcategories were reported as high-risk, the whole category of blinding bias was considered high-risk, and if all were at low risk, the whole category was considered low-risk; otherwise, the blinding bias was reported as unclear.
Some other RoB assessment items were rarely reported; for instance, funding bias, intention to treat bias, sample size bias, for-profit bias, and power calculation bias. Due to these items’ low frequency, these kinds of biases were not extracted in this study. Additionally, a new variable was defined to aggregate the assessments’ full results, named the “Overall Risk of Bias” (ORoB). The criteria for determining the ORoB were the same as those of the “Overall Bias” assessment in Cochrane’s RoB tool -version 2 (18). If any of the bias domains were considered high-risk, the ORoB was considered high. In the absence of high-risk biases, unclear-risk-assessed biases were perceived in the same way; otherwise, the ORoB was considered low (19). The RCTs that appeared in more than one Cochrane review were excluded if their RoB assessments were not consistent.
In including RCTs in this study, several groups were considered, namely, quasi-experimental (controlled before-after, interrupted time series, and non-randomised designs), parallel RCT, cross-over RCT, cluster RCT, split-mouth RCT, and repeated-measures study designs.
Finally, to scrutinise the Cochrane review authors’ subjectiveness and bias of judgement, the studies were distinguished by RoB domains. Cochrane reviews that included the same studies, either with the same or different RoB assessments, were listed and signposted in different categories based on the included titles.