The diagnosis and prognosis of patients with acute infections and/or suspected sepsis is an ongoing clinical challenge. The clinical presentation in sepsis is often non-specific and current diagnostic tools have limited sensitivity and/or require a long time to obtain results and do not assess prognosis or etiology 6,8,25. This manuscript presents results of a large registrational clinical trial establishing performance criteria for the TriVerity Test. The TriVerity Test combines innovative transcriptomic measurement of 29 host response mRNAs and interpretation by machine-learning algorithms. It rapidly provides simultaneous results for presence (infected or not), type (bacterial or viral), and illness severity in patients with suspected acute infection and suspected sepsis. Clinical actionability is made possible by high test accuracy, intuitive result readouts as scores that fall into defined interpretation bands and the test’s rapid turnaround time of approximately 30 minutes from an easy-to-use cartridge-based instrument.
The present SEPSIS-SHIELD study demonstrated very high overall accuracy of the TriVerity Test with AUCs of 0.83 and 0.91 for identifying bacterial and viral infections, respectively, compared to the reference standard of clinically adjudicated infection status. We then calculated specificity and sensitivity of each of the TriVerity interpretation bands that innovatively permit to rule-in and rule-out bacterial and viral infections. Importantly, we simultaneously observed very high specificities for the diagnosis of bacterial and viral infection (95.5% and 98.6%, respectively) and very high sensitivities for ruling-out bacterial and viral infections (97.2% and 95.5%, respectively). When incorporating prevalences, these findings translated into high probabilities for ruling-in or ruling-out of bacterial and viral infections. The clinical value of TriVerity results is further supported by the finding that most results fell into the highly actionable but not the less actionable Moderate (middle) band.
The TriVerity Tests offers the potential to assist the ED clinician to make more informed decisions about several clinically relevant and common decisions including the administration of antibiotics, helping to select those patients who would benefit from anti-infective therapy while mitigating anti-biotic overuse, thereby assisting in reducing common side effects of antimicrobials and combating the further emergence of antimicrobial resistance 26,27. Notably, TriVerity met the expert consensus target product profile for a diagnostic assay to differentiate between bacterial and non-bacterial infections, which can reduce antimicrobial overuse 28. In addition, to the best of our knowledge, the TriVerity Test is the first direct “biomarker” for detection of viral illness; recently introduced protein-based host response tests only distinguish bacterial vs. non-bacterial etiologies not allowing to fully distinguish viral infections from noninfectious inflammation or co-infections 29,30. Specific and accurate identification of patients with viral infections will be adjunctive to other approaches for ED clinicians in helping them to decide on specific viral diagnostics, antiviral treatment and isolation measures to prevent the spread of infection. Further, the simultaneous presentation of TriVerity Bacterial and Viral scores may reinforce the impact of a single bacterial result in support of antimicrobial stewardship. We also found that TriVerity viral results are stable across different viruses, including SARS-CoV-2. Of importance, TriVerity can also detect bacterial-viral coinfections. While the prevalence of coinfection is low in the ED, ruling out bacterial co-infections is an important feature to allow withholding antibiotics, especially when radiographic or other clinical features introduce diagnostic uncertainty, and during pandemic threats 31,32. Finally, TriVerity results indicating low probabilities of bacterial and viral infection allow clinicians to focus their efforts on non-infectious diagnoses thereby avoiding ordering of expensive pathogen identification tests and motivating clinicians to find the non-infectious cause of illness.
While the overall accuracy of the bacterial and viral TriVerity results was lower using the forced vs consensus adjudication (an expected finding as cases with uncertain infection status are included in the analysis), the accuracy remained high. It is important to remember that in the context of this study accuracy was determined post-hoc using the best possible reference standard, clinical adjudication 33. In daily practice in the ED it is envisioned that TriVerity results will be used in addition to provider assessment based on medical history and clinical presentation which is known to be far from accurate.
Bacterial infections were most frequent (61%) compared to viral infections (22.0%) and co-infections (1.6%) while 14.3% of participants did not have an infection. A high proportion of patients were immunosuppressed, but we did not observe significant differences in the accuracy of TriVerity in immunocompetent vs. immunocompromised patients. These findings indicate that the TriVerity Test can be safely used across patients with diverse infections and immune statuses, an important feature for patients presenting to the ED. Importantly, the performance in immunocompromised patients has not been determined in clinical studies for other host response tests as these were validated in patient cohorts excluding immunosuppression.
Overall, TriVerity AUROCs for detecting bacterial infections were significantly higher than those for commonly used biomarkers including PCT, CRP and white blood cell counts. These findings are remarkable as the results of these biomarkers were provided to clinical adjudicators when adjudicating the infection status and thus may have introduced bias favoring the accuracy results for these biomarkers. Notably, the TriVerity Test as the first direct biomarker for the diagnosis of acute viral infections also detected influenza and SARS-CoV-2 infections during the present study (enrollment into SEPSIS-SHIELD occurred before, during, and after the COVID pandemic). The TriVerity Test’s ability to provide both highly accurate bacterial and viral diagnoses could contribute to more appropriate management of patients with suspected infections. The rapid and early identification or ruling-out of bacterial and viral infections could assist in the timely administration of antibiotic or antiviral drugs (or withholding of antimicrobials) supporting antimicrobial stewardship, (cohort) isolation to combat the spread of infections and the appropriate ordering of follow-up tests.
The early identification and treatment of suspected sepsis are critical34 but there is disagreement among major societies regarding exact guidance and timing for optimizing clinical care. For example, the Centers for Medicare & Medicaid Services (CMS)’s SEP-1 bundle, and the Surviving Sepsis Campaign guidelines strongly recommend the rapid administration of antibiotics to patients with suspected sepsis but physician-based societies (e.g., the Infectious Diseases Society of America) have argued that they do not have adequate tools to judge who has sepsis. These challenges contribute to overtreatment, and substantial costs and morbidities associated with patients given unnecessary antibiotics5,35 including the rise of antimicrobial resistance. At the same time (and driving some of the practice patterns), occult sepsis continues to be overlooked in a significant proportion of ED patients. While qSOFA greater than or equal to 2 conveys a high percentage mortality, further classification of patients with qSOFA 0–1 is needed as the percentage mortality is low, but because the group is larger, still can account for 30% of sepsis deaths36. In the present study, the TriVerity Illness Severity score demonstrated an overall AUC of 0.85 for the prediction of “ICU-level care” within 7 days with very high specificity (98.7% for the Very high band) to rule-in severe illness while also showing high sensitivity (91.8% for the Very low band) for ruling-out severe illness. The TriVerity Test also demonstrated high accuracy in ruling in and ruling out ICU transfer. In a post-hoc analysis, TriVerity was able to fine-tune the risk prediction based on clinical scores, i.e., qSOFA with conjectured use of TriVerity dramatically improved accuracy in those patients clinically at low risk but at significantly increased risk of severe illness (by 7-fold). TriVerity also correctly identified those patients with significantly lower risk that had clinically been identified as high risk. The TriVerity Illness Severity score also provided additional information to aid in identification of patients with sepsis.
Taken together, TriVerity could assist in the fine-tuning of patient management in the ED, with results offering potential for improved clinical decision making, including disposition decisions. Remarkably, the TriVerity severity results not only predicted the short-term outcome of “ICU-level care” but also the longer-term outcome of 28-day mortality (the secondary prognostic endpoint of the study, data not shown), thereby further strengthening the validity of TriVerity illness severity results.
Results of the current study, along with findings of other studies21–24 point towards broad generalizability of our findings. First, demographics of enrolled participants were representative of common US ED patients with respect to age, sex, race and ethnicity, underlying conditions, immune status and clinical outcomes. While PCT performs ‘moderately’ well in identifying bacterial infections, results have historically been less accurate in Black/African Americans and Hispanic subgroups37. Our finding of significantly superior accuracy of the bacterial TriVerity result across White, Black/African American and other races in the current study offers the opportunity for improved accuracy in the diagnosis of bacterial infections in minority populations where PCT is known to underperform.
Our study has limitations. First, without a true gold standard, we used clinical adjudication as the reference standard to determine the patient infection status33. As expected, the accuracy of the TriVerity Test for diagnosing bacterial and viral infections was highest in the consensus adjudication population (certain infection status). While consensus adjudication appears to be the most accurate comparator to assess infection status, it removes a substantial fraction of patients. Adding patients with uncertain infection status (forced adjudication, Supplementary Table 2) allowed accuracy to be calculated across the entire cohort of SEPSIS-SHIELD. Second, the prevalence of bacterial (58%) compared to viral (20%) infections and non-infectious diseases (19%) may appear high. However, the observed high bacterial prevalence was likely driven by a) the broad inclusion criteria resulting in approximately 50% of all patients having abdominal, skin and soft tissue or urinary tract diseases (rarely caused by viral pathogens), and b) the seasonality of bacterial vs. viral infection (i.e. respiratory tract infections). Depending on local epidemiology and setting, TriVerity will demonstrate varying positive and negative predictive values in different settings. However, SEPSIS-SHIELD enrolled many patients and over cold and other seasons, and patient characteristics were characteristic of a US ED population. Therefore, we believe that the validity of the results reported here is not affected by these factors. Furthermore, the development of the machine learning-based TriVerity classifiers was based on a large number of datasets from highly diverse cohorts of patients with suspected infections and/or suspected sepsis20. Lastly, the percentage of patients with severe illness (“ICU-level care”) within 7 days was only 10.9%, lower than expected; thus, while results of the TriVerity Illness Severity score indicate high accuracy for the prediction of “ICU-level care”, future validation studies are warranted. Also, we have previously shown consistently high accuracy among patients admitted to the ICU17,38,39. Interventional and randomized studies to demonstrate the clinical benefit of TriVerity for diagnosing bacterial and viral infections and the prognosis of illness severity in the ED are in the planning or preparation phase.
Importantly, our statistical analysis approach in SEPSIS-SHIELD ensured inclusion of all patients as equivocal adjudication was not allowed (adjudication categories were Yes, Probable, Uncertain or No) and included all patients with results that fell into the (middle) Moderate TriVerity band. This strategy is important because several studies have been published in which statistical analysis excluded both patients with uncertain adjudication (a clinically critical “gray zone” population) and those in the middle (e.g., equivocal) interpretation bands thereby unfairly over-stating the test overall accuracy40–43.
In conclusion, the TriVerity Test provides rapid, highly accurate and actionable results for the diagnosis and prognosis of patients with suspected acute infection and sepsis, serving a major unmet medical need. The test is highly innovative as it can detect bacterial infection, viral infection, bacterial-viral coinfections and rule out bacterial and viral infections, thereby showing superior accuracy compared to routine biomarkers. TriVerity can also predict the severity of illness, and combined with clinical findings, TriVerity adds granularity and important potential clinical utility for predicting severe infection. Together, these properties together suggest that TriVerity can improve personalized management of patients with suspected acute infections and suspected sepsis for improved overall healthcare outcomes.
Methods
Study design
The “TriVerity in the Diagnosis and Prognosis of Emergency Department Patients with Suspected Infections and Suspected Sepsis” (SEPSIS-SHIELD, clinicaltrials.gov NCT04094818) was conducted between 02 March 2020 and 28 May 2024. SEPSIS-SHIELD was a prospective, non-interventional, minimal-risk study to analyze gene expression and other laboratory data from biological samples collected from patients presenting to the ED with suspected acute infections with at least one abnormal vital sign, or patients with suspected sepsis of any cause with at least two abnormal vital signs and a blood culture order (see below for precise enrollment criteria). Study sites included emergency departments (EDs) of community and academic hospitals and were located at 21 geographically diverse locations throughout the United States and one site in Europe (Supplementary Table 1). The enrollment period was divided into a “FROZEN” phase (02 March 2020 to 16 Feb 2023) and a “FRESH” phase (08 Dec 2023 to 28 May 2024). During the “FROZEN” phase of the study, blood samples were frozen and sent to one of two reference laboratories to perform the TriVerity Test after thawing the frozen samples. During the “FRESH” phase of the study, the TriVerity Test was performed on freshly obtained blood samples (no freezing) at the enrolling site.
Patients, sample, and data collection
Eligibility included ED patients who met all of the following inclusion criteria: 1) age ≥ 18 years, 2) suspected acute infection, e.g., respiratory, urinary, abdominal, skin and soft tissue infection, meningitis/encephalitis, or any other infection and 3) at least one vital sign change (heart rate > 90 beats/minute, temperature > 38°C or < 36°C, respiratory rate > 20 breaths/minute or PaO2 of < 60 mmHg or SpO2 < 90%, systolic heart pressure < 100 mmHg, altered mental status per clinical exam) OR suspected sepsis of any cause defined by a blood culture order by the treating physician and at least 2 vital sign changes, 3) able to provide informed consent or consent by a legally authorized representative. Patients were ineligible if they met any of the following exclusion criteria: 1) Patient-reported treatment with systemic antibiotics, systemic antiviral agents, or systemic antifungal agents within the past 7 days prior to the ED visit (use of antiviral treatment for HIV, hepatitis B and hepatitis C, topical antibiotics, topical antivirals, or topical antifungal agents, anti-herpes prophylaxis, peri-operative [prophylactic] antibiotics and a single dose of antimicrobials during the present ED visit [< 10 hours before blood draw] did not result in exclusion), 2) patients who were receiving palliative or hospice care, or those who were receiving limited interventional care, 3) prisoners or those mentally disabled or unable to give consent (consent by legally authorized representatives were accepted), 4) patients who were receiving experimental therapy or who were already enrolled in an interventional clinical trial in which a patient received some type of intervention, which could include but was not limited to investigational drugs, medical devices, or vaccines and 5) patients previously enrolled in this clinical trial. Of note, patients with any form of immunosuppression were eligible for SEPSIS-SHIELD.
All patients had routine samples collected as per the standard of care and all patients were managed as per the standard of care independent of the study conduct. The study intervention consisted of blood collection via venipuncture (2.5 ml whole blood into PAXgeneⓇ Blood RNA Tube (PreAnalytix), 5 ml of whole blood for central laboratory testing of serum C-reactive protein and PCT) and a nasopharyngeal swab collection from those patients with suspected upper respiratory tract infections. Patients were followed for up to 28 days, and those discharged earlier were contacted via phone-call to collect follow-up information (re-admission etc.). Data collection occurred from the day of presentation in the ED (covering 7 days before enrollment) to the follow-up phone call after day 28 using a case report form. Data collected included detailed information on demographics, medical history, clinical, laboratory, and imaging findings as well as information related to management including treatment with antibiotics, ICU-level treatment etc. All data were stored pseudonymized in a secure database.
Endpoints
The primary endpoints for the bacterial etiology and viral etiology were determined by independent adjudicators following a transparent and standardized clinical adjudication process that was developed for the SEPSIS SHIELD trial by a multidisciplinary team of physicians and laboratorians described elsewhere (Whitfield et al., DMID 2024). In brief, two independent physicians reviewed comprehensive clinical, laboratory and other patient information to adjudicate the presence or absence of bacterial and viral infections into Yes, Probable, Unlikely and No categories (no equivocal adjudication allowed); these adjudicators were randomly chosen from a pool of 12 ED physicians. Discordant cases were resolved by one of two expert physicians (experienced and involved in generating the adjudication protocol) who were blinded to the initial reviewer results. Cases adjudicated as Yes and No by the adjudicators (subgroup with certain infection status) formed the consensus adjudication cohort presented in the main body of this report. Cases adjudicated as Probable or Unlikely were forced into categories of Yes/Probable and No/Unlikely to form the forced adjudication cohort (all patients but including uncertain and certain adjudications). The severity outcome of patients required “ICU-level care” was met if a patient received any of the following during hospitalization: mechanical ventilation, vasopressor use, and/or renal replacement therapy.
TriVerity Test
The TriVerity Test (formerly HostDx or InSep, pending FDA clearance) is a gene expression profiling assay that quantifies the relative expression of 29 host response genes from 2.5 ml whole blood collected in a PAXgene Blood RNA tube. The TriVerity System (Supplementary Fig. 2A) is comprised of the TriVerity Cartridge and the Myrna Instrument with embedded software and proprietary pre-processing and machine-learning classification algorithms20 that process the data and deliver the results. The system is designed for single-sample testing. The TriVerity Test generates three scores corresponding to 1) the likelihood of a bacterial infection, 2) the likelihood of a viral infection, and 3) the severity of the patient’s illness. They fall into actionable interpretation bands ranging from Very high to High, Moderate, Low, and Very low (Supplementary Fig. 2B). Analytical and other details of the TriVerity System are described elsewhere (Rasania et al., in preparation).
Statistical Methods
The analysis population consisted of all participants who provided consent, met all inclusion criteria, met no exclusion criteria and had their PAXgene blood sample collected and successfully processed (resulting in three TriVerity scores). The overall analysis population was further divided into diagnostic and prognostic analysis populations with additional requirements. The diagnostic endpoint analysis population included participants who had a completed adjudication for bacterial and viral infection to define the ground truth defined as either a certain (Yes or No) or uncertain (Probable or Unlikely) infection status. The diagnostic population was further classified as Consensus (only contains participants with a certain [Yes or No] adjudication) and Forced (all patients adjudicated as [Yes or Probable] vs. [No or Unlikely]). The primary prognostic endpoint analysis population included participants who had information available on the presence of severe illness (acute need for mechanical ventilation, vasopressor, and/or RRT) within 7 days. The secondary prognostic endpoint included those participants that met the primary endpoint above and/or 28-day mortality.
Each of the bands for the TriVerity Test were assessed in terms of likelihood ratio, sensitivity, specificity, and predictive values (post-test probabilities). The calculation is dependent on the band (whether intended as a rule-in band with a case being a true positive, or a rule-out band with a case being a false negative). For continuous variables (e.g., age, white blood cells), results were summarized with the numbers of observations, means, standard deviations, medians and ranges and confidence intervals. The Fisher's exact test was used to determine the significance of differences in the need of “ICU-level care” across patients with different infection types. P-values for differences between the AUROCs of the TriVerity Test and biomarker concentrations or biomarker counts, clinical scores (qSOFA), and differences across races were calculated using DeLong’s test, whereas p values comparing immunocompromised and immunocompetent patients as well as COVID-19 vs non-COVID-19 patients were calculated using bootstrapping. 80% confidence intervals are given where appropriate. Statistical analyses were performed in SAS version 9.4, or R software version 4.2.
Ethics statement
This study was approved by local or central Institutional Review Boards or Independent Ethics Committees and written permission was obtained from each patient. The study was conducted with the highest respect for the individual patients and in accordance with the protocol, the ethical principles that have their origin in the Declaration of Helsinki, the informed consent regulations stated in Title 21 Code of Federal Regulations (CFR), Part 50, International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH), GCP (E6) § 4.8, as well as all applicable local and FDA regulations.
Data availability