Protocol and registration
This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) extension statement for reviews incorporating network meta-analyses [9]. The protocol has been registered in protocols.io (Protocol integer ID 49629) [10].
Inclusion criteria
We included randomized controlled trials (RCTs) reported in English and Japanese, comparing two of the following four strategies: SOT (nasal cannula, facemask, and venturi mask with limitless flow rate), NPPV (mask type, ventilation mode, and methods of weaning were not limited), HFNO (flow rate and fraction of inspired oxygen were not limited), and IMV (mechanical ventilation via endotracheal intubation, not tracheostomy).
This review included adults (age ≥ 18 years) with AHRF, defined by any of the following criteria: new onset (< 7 days) of clinical signs (e.g., tachypnea), radiological signs (e.g. chest X-ray opacities), and hypoxemia. Hypoxemia was defined as the ratio of arterial oxygen partial pressure to fractional inspired oxygen below 300, arterial or percutaneous oxygen saturation < 94% in room air, PaO2 < 60 mmHg in room air, or < 80 mmHg with oxygen.
The primary outcome was the incidence rate of pneumonia (ventilator-associated pneumonia (VAP), aspiration pneumonia, and nosocomial pneumonia were not limited). The secondary outcome was the incidence rate of VAP (VAP was defined in each study).
Exclusion criteria
Randomized crossover, cluster-randomized, or quasi-experimental trials were excluded. The current meta-analysis excluded studies in which more than half of the patients had congestive heart failure, acute exacerbation of chronic obstructive pulmonary disease (COPD), asthma attack, hypercapnia (PaCO2 > 50 mmHg), respiratory failure due to post-extubation or trauma, post-surgical status, or do-not-resuscitate orders. We also excluded the studies which had limited intervention in the emergency department or pre-hospital care.
Search strategy
Databases used for the search of eligible trials were The Cochrane Central Register of Controlled Trials, MEDLINE via PubMed, EMBASE, and Ichushi, a database of Japanese research papers. Because this systematic review was planned for ARDS clinical practice guidelines 2021 in Japan [11], we also included studies on clinical questions about non-invasive oxygenation strategies in the guideline, using manual search. A literature search was performed on May 30, 2021. The terms used for database search showed in Appendices.
Study selection and data extraction
At the first screening, two of the five reviewers (HO, TM, SH, SK, and MS) screened the title and abstract. At the second screening, the full text for relevant studies was analysed and data were independently extracted from the included studies into standardized data forms. Disagreements were resolved by discussing with one of the three reviewers not involved in screening the studies. We also asked the original authors for additional details when necessary. After identifying studies in the second screening, we extracted the following study characteristics: methods, participants, interventions, and outcomes.
Network geometry
Network diagrams were constructed using the Confidence In Network Meta-Analysis (CINeMA) web application [12], to show the number of studies and patients included in the meta-analysis. We represented network geometry that treatments by nodes and head-to-head comparisons by lines connecting these nodes. The size of the node is proportional to the number of patients, while the thickness of the lines is proportional to the number of studies evaluating each treatment.
Quality assessment
The risk of bias of outcomes in the eligible studies was independently assessed by two of the five authors using the Cochrane Risk of Bias tool 2.0 [13] for the following seven domains: (a) random sequence generation, (b) allocation concealment, (c) blinding of participants and personnel, (d) blinding of outcome assessors, (e) incomplete outcome data, (f) selective outcome reporting, and (g) other sources of bias. Each domain of bias was graded as either “low risk,” “unclear risk,” or “high risk.” If there was a discrepancy between the two reviewers, an agreement was reached through discussion. Discrepancy checks were resolved by discussion with a third reviewer, as necessary. Each domain was evaluated in three categories: high risk, low risk, and some concerns.
Methods of direct comparison meta-analysis
A pairwise meta-analysis was performed using Review Manager version 5.3 [14]. Forest plots were used for meta-analysis, and the effect size was expressed as a risk ratio (RR) with 95% confidence interval (CI) for categorical data. Outcome measures were pooled using a random-effects model to analyse study-specific effects in the measures. A two-sided p-value < 0.05 was considered statistically significant for all analyses.
Methods of network comparison meta-analysis
An NMA was performed using a frequentist-based approach with multivariate random-effects meta-analysis, and the effect size was expressed as the RR (95% CI) using the CINeMA web application [12]. CINeMA is based on the framework that was developed by Salanti et al. [15] and modified by Nikolakopoulou et al. [16]. Six domains that influence the level of confidence of NMA results are: (a) within-study bias, (b) reporting bias, (c) indirectness, (d) imprecision, (e) heterogeneity, and (f) incoherence. The covariance between two estimates from the same study indicated the variance of data in the shared arm, as calculated in a multivariable meta-analysis performed for an NMA [17]. Transitivity was assessed in the incoherence domain using CINeMA. We constructed forest plots of the RR with 95% CI for each treatment strategy in the network.
Ranking plots (rankograms) were constructed based on the probability that a given treatment had the highest event rate for each outcome. The surface under the cumulative ranking curve (SUCRA), which is a simple transformation of the mean rank, was used to determine the treatment hierarchy. We calculated values of the SUCRA statistic using the mvmeta command in Stata 15.1 (StataCorp LLC, College Station, TX, USA) [18]. SUCRA is a percentage that shows how much effectiveness a treatment archives in comparison with a theoretical treatment that is always the best. The larger the SUCRA value, the better the rank of the treatment [19].
Assessments of the certainty of evidence
We evaluated the indirectness, classified as “no”, “some”, or “major” concern, of each study included in the NMA based on its relevance to the research question. The study-level judgments can be combined with a percentage contribution matrix. The approach to imprecision involved comparing the range of treatment effects included in the 95% CI with the range of equivalence. We assessed the heterogeneity of treatment effects for a clinically important risk ratio of < 0.8 or > 1.25 in prediction interval. Study heterogeneity among trials for each outcome was assessed by visually inspecting forest plots and with an I2 statistic to quantify inconsistency [20]. Publication bias was visually assessed using a funnel plot [19]. The global inconsistency test with a fitting design-by-treatment model was used to identify the disagreement between the direct and indirect estimates as a measure of inconsistency [21]. The studies were categorized based on the p-value of the global design-by-treatment interaction test: “major concerns” (< 0.05), “some concerns” (0.05–0.10), and “no concerns” (> 0.10). Studies were also categorized as showing “major concerns” if the design-by-treatment interaction test statistic could not be computed due to the absence of closed loops in the network. The network estimate was rated considering the risk of bias, indirectness, imprecision, heterogeneity, publication bias, and inconsistency [22].