The rapid identification of sepsis serves as the basis for its successful management. According to the current recommendations of the Surviving Sepsis Campaign, the SOFA score should be used as a predictive tool for the detection of sepsis as well as for the risk stratification of critically ill patients. In addition, the qSOFA score has been introduced for the identification of septic patients outside of the ICU [7, 21, 22]. However, to our knowledge, both scores have not been evaluated in the context of surgical IMCU patients. Therefore, this is the first study comparing the predictive value for presumed sepsis of the SOFA and qSOFA scores, as well as the 1992 defined SIRS criteria, in a large cohort of 13,780 surgical IMCU and ICU patients of a tertiary university hospital.
Overall, among encounters with suspected infection in IMCU patients, none of the analyzed scoring tools showed sufficient predictive validity for severe infection (defined as the use of broad-spectrum antibiotics), whereas the qSOFA score was able to predict mortality in a sufficient manner. Interestingly, even though the assessment with the historical SIRS criteria does not meet the current practice guidelines, they were more predictive than the SOFA score within the IMCU patient cohort. Furthermore, among ICU patients as well as patients who underwent a combined IMCU and ICU treatment, all analyzed scoring systems failed to provide sufficient validity for the prediction of infection and mortality. Only in patients who underwent a combined IMCU and ICU treatment the SOFA score was able to adequately predict mortality.
At first glance, these study results might be surprising. However, in comparison to previous findings, the performance of the qSOFA score and the SIRS criteria remain agreeable. The qSOFA score was first developed and validated by Seymour et al., who analyzed 148,907 unselected patient encounters with suspected infection, consisting of a validation cohort of 7,932 ICU and 66,522 non-ICU patients. With the exception of the SOFA score, the predictive value of the qSOFA and SIRS criteria could be matched to our study results within the ICU cohort (AUCROC SOFA: 0.74 vs. 0.52; AUCROC qSOFA: 0.66 vs. 0.63; AUCROC SIRS: 0.64 vs. 0.63) [7]. Their findings have been validated in several studies featuring large numbers of patients (Table 4), resulting in a varying performance of the mentioned scores. However, it has to be stressed that originally Seymour et al. aimed to validate the qSOFA and SOFA scores as predictors for mortality and not for the identification of sepsis. The Sepsis-3 definition indicates that, due to their predictive value for mortality, both scores can be used for sepsis risk stratification (either at the ICU or non-ICU), but also emphasizes that the underlying data has derived from retrospective studies and requires further validation. However, until now, no prospective data, with sufficient numbers of patients, is available.
Lo et al. performed a literature review and retrospective data analysis of 380,920 patients, demonstrating an AUCROC of 0.68 for the predictive value of in-hospital mortality for the qSOFA score, which is in line with our study findings in surgical ICU and IMCU patients [21]. Furthermore, a meta-analysis of 229,480 patients compared the qSOFA score and SIRS criteria for their ability to predict patient mortality, and revealed only a slightly better performance of the qSOFA score, which supports the findings of our study [23]. However, some studies revealed a high power for the prediction of mortality: Kovach et al. analyzed hospital mortality in a retrospective data set of 3,749 surgical and medical ICU patients with suspected infection, while Zhang et al. investigated retrospectively 5,109 cardiac surgical patients, with both studies resulting in AUCROC > 0.8 for the prediction of mortality by using the SOFA and qSOFA scores [22, 24]. However, it must be highlighted that, contrary to our approach, the patients of Kovach´s study were adjusted for a baseline risk factor for death, which increased the predictive quality of the SOFA score, while Zhang et al. only included cardiac surgical patients, which are hardly comparable with the sources of systemic inflammation in our study. During cardiac surgery, systemic inflammation is mainly induced by cardiopulmonary bypass, which leads to strong activation of the inflammatory response through the blood´s foreign surface contact with the components of the heart-lung machine, as well as through ischemia/reperfusion injury [25]. Contrarily, local surgical trauma is causative for the onset of inflammation during non-cardiac surgery.
Even though qSOFA and SOFA scores are widely accepted as tools for the identification of septic patients, they failed to predict suspected infection in each individual group of patients in our study. These findings are supported by Krebs et al., who also evaluated the qSOFA and SOFA scores as well as the SIRS criteria in 1,942 prospective patient days within a cohort of surgical trauma ICU patients, concluding that all scores failed to predict the development of new infections [18]. But, also in an out-of-ICU setting, a failure of the qSOFA score (and SIRS criteria) has already been described in a collective of patients attending the emergency room (n = 1,045) [26]. Moreover, another large retrospective analysis failed to prove a high predictive power of the qSOFA score and the SIRS criteria in patients admitted to the emergency department [27].
These opposing results might be partially explainable, as already discussed above, by the choice of the study population, which might strongly influence the study results because only postsurgical patients were investigated in our study, in contrast to medical and surgical patients in the underlying study. Further, the variable predictive validity between the studies might be caused by the differences in the study designs. Considering the original publication of Seymour et al., the lower predictive capacity of the SOFA score in our study might be caused by the varying definition of suspected infection. While it was defined as the combination of antibiotics and body fluid cultures by Seymour et al., the administration of broad-spectrum antibiotics was used in our approach. It has to be noted that the prescribing behavior of antibiotics varies between physicians depending on their clinical experience, qualification, and specialty. Charani et al. compared the antibiotic prescribing between medical and surgical specialties. Besides more frequent and longer prescription, antibiotics were more likely to be escalated in surgical patients [28]. A recent systemic review offers a potential explanation for these findings by identifying nine determinants that influenced antibiotic prescription behavior including the fear of risking an adverse outcome [29]. Surgical patients are challenging when it comes to identify infectious complications and the consequences of sepsis are more devastating in these patients which potentially leads to a more liberal application of broad-spectrum antibiotics [30, 31]. This might offer an explanation for the low specificity of the analyzed scores for detecting a presumed sepsis. Furthermore, even in an isolated analysis of studies that only investigated surgical patients, the predictive performance varies strongly: Falcao et al. analyzed 3,008 surgical ICU patients and showed a sufficient predictive validity of the SOFA score regarding mortality (AUCROC of 0.74) [32]. Similar results are published by Basile-Filho et al., who revealed an AUCROC of 0.79 by using the SOFA score for the prediction of mortality within 847 surgical ICU patients [33]. Contrarily, Mungan et al. showed a lower predictive validity of 0.63 of the SOFA score in surgical ICU patients [34]. Although authors of these studies described their population as "surgical patients“, it must be highlighted that their calculations comprised all kinds of surgical patients, independently of their risk for infection, including those without suspicion of infection. By contrast, our study only focused on the investigation of postsurgical patients with suspected infection. In our opinion, this issue is of high relevance because the majority of postsurgical patients following major surgery regularly show clinical signs of systemic inflammation such as tachycardia, fever, and tachypnea, which commonly represent signs of a surgery-induced systemic inflammation rather than an infection. For this reason, it is not only challenging to discriminate between postsurgical sterile systemic inflammation and infection, but the predictive ICU sores might also become distorted into false positive results. This may explain the high sensitivity but low specificity of the qSOFA score and SIRS criteria because both systems include only clinical criteria for easy assessment. Since these criteria are often fulfilled during the postsurgical phase, the chance that they are truly positive is high (sensitivity). On the other hand, this leads to a low rate of true false cases (specificity). Since the SOFA score consists of much more detailed intensive care variables than the qSOFA score and the SIRS criteria, the specificity is higher, but sensitivity remains low. These arguments are in accordance with the findings of Gando et al. as well as Krebs et al., who demonstrated that the SIRS criteria, SOFA score, and qSOFA score were not able to predict sepsis in the emergency department or surgical ICU [18, 26].
These limitations of the ICU scores are of high interest for their use on surgical IMCUs because of the increasing demand of IMCU capacity. Therefore, the importance of the surgical IMCU, as a bridge to the normal ward, is rising. Patients attending the IMCU commonly represent those surgical patients at moderate to high risk of developing postsurgical complications. Analogous to ICU patients, the rapid identification of infectious complications is altered by surgery-induced signs of systemic inflammation, underlining the need for specific IMCU scores. Lacking studies that focus on surgical IMCUs, other high-dependency units (HDUs) (but not ICUs) have to be analyzed for the interpretation of our study results. Innocenti et al. analyzed 3,311 patients admitted to HDUs and demonstrated that the SOFA score, in opposition to our results, showed a good discriminatory ability for HDU mortality [35]. However, contrary to our approach, no cut off values for SOFA scores were used, and no postsurgical patients were included. In our study, the prediction of mortality was sufficient using the qSOFA score in IMCU patients. Another study showed that these scores are also not able to predict infection in the emergency room [26]. Based on the findings of our study, the use of the qSOFA score as a predictor of mortality can be supported, while its predictive power for the detection of suspected infection can be doubted in postsurgical IMCU patients, which might be caused by surgery-induced systemic inflammation.
However, due to the retrospective study design, further prospective studies that include high numbers of postsurgical IMCU patients are necessary to validate these findings. Due to the fact that most critically ill patients are regularly transferred to the IMCU during their medical treatment, a subgroup of these patients was created. The increased APACHE II score reflects the serious illness of the included patients. Since clinical scores were not able to distinguish for suspected infection in this subgroup, severity of disease seems not to improve the quality of these scores. While adequate prediction for mortality was calculated with the SOFA score, this could indicate its better quality in critically ill patients. Since these patients are missing in the ICU subgroup, this could also explain our lower results for the SOFA score in the ICU.
Nevertheless, our study features some limitations. First, this retrospective analysis is not able to draw conclusions regarding the underlying causalities. Second, due to the retrospective design, no sample size calculation was performed. Third, lacking of more specific alternatives, the administration of broad-spectrum antibiotic agents was used as a surrogate for the diagnosis of suspected infection. While the clinical symptoms and inflammatory parameters are physiologically altered by the surgery, body fluid cultures result in negative samples in a majority of cases (e.g., due to the perioperative antibiotic treatment) [29, 36, 37]. Furthermore, even if sepsis was assessed by intensivists, its diagnosis remains subjective [38]. Nonetheless, it has to be highlighted that the administration of a broad-spectrum antibiotic agent serves only as a surrogate for the true presence of sepsis. Fourth, the RAAS was used as a surrogate parameter for GCS < 15 in the absence of the GCS, which is problematical since the qSOFA score was developed and validated with the use of GCS.