We assessed a total of 467 clinical files belonging to patients that were hospitalized at Sociedad Española de Beneficencia and Hospital Español, from March 12th 2020 to August 1st 2022. All files were analyzed, and upon assessment 422 were found to be suitable for analysis. 255 files were allocated to allow for algorithm design and 167 were used to validate the algorithms (Supp. Figure 1). The only criteria for such allocation was to use the files belonging to Hospital Español in the design of the algorithms, while the files from Sociedad Española de Beneficencia were used to test the scales with a different population. The patients whom contributed with the data for algorithm design were unvaccinated against SARS-CoV-2, while only 26% of the patients that provided data for the algorithms validation had already received such treatment.
Of the 255 clinical files that we used to calculate the algorithm 175 (74.5%) belonged to patients that were retrospectively found to have a justifiable hospitalization, while 59 (25.5%) did not develop characteristics that made hospitalization mandatory over their whole stay. Moreover, 125 (49.6%) patients required IC and 79 (31.1%) died at the hospital.
Given that severely affected chest tomography findings (% inf) [21], C-reactive protein (CRP), d-dimer, neutrophils, lymphocytes, lactate dehydrogenase (LDH) [22], procalcitonin, medium arterial pressure (MAP), creatinine, leukocytes, aspartate aminotransferase (AST) [23, 24], ferritin, oxygen saturation (sO2) [25], advanced age and comorbidities [26] have been associated with COVID-19 progression, the exact values of these markers were extracted from the complete clinical files. Upon gathering the laboratory data for the first 24 hrs of hospitalization, the OR for each of the markers was calculated in relation to HR (Supp. Table 1), ICR (Supp. Table 2) or mortality (Supp. Table 3). We found four significant associated markers (Kirby index, sO2, CRP and LDH) with HR, 10 with ICR (d-dimer, neutrophils, % inf, age, Kirby index, sO2, CRP, ferritin, LDH and AST) and 11 with mortality (creatinine, d-dimer, neutrophils, % inf, age, MAP, Kirby index, sO2, CRP, ferritin, > 1 comorbidity and LDH).
The markers with significant associations were plotted into a heat map and their Pearson’s correlation coefficient was calculated (Fig. 1a, c and e) and used to determine the relative weight, or mathematical function, of each variable into each of the three algorithms. Only four variables had a Pearson’s correlation index ≥0.20 in relation to each outcome, and thus were considered as important for the development of the algorithms, being Kirby<200, LDH>211, CRP>120 and sO2<80 for the prognostic of hospitalization; Rx>14, Kirby<200, CRP>120, and LDH>400 for the prediction of ICR; and finally age>60, Kirby <150, CRP>120 and Rx>15 for mortality (Fig. 1b, d and f).
To calculate the OP for each patient , the Pearson’s index of each variable (Fig. 1b, d and f) was added each time a particular patient presented an abnormal level of a particular marker, and then both control (outcome negative) and experimental (outcome positive) patients’ values were used to calculate the area under the ROC curve (AUROC). The COVID-hospitalization outcome prognostic (COVID-HOP) scale had an AUROC of 91% (CI 0.8725 to 0.9482 at 95%, p<0.0001), and both the COVID-intensive care outcome prognostic (COVID-ICOP) and the COVID-mortality outcome prognostic (COVID-MOP) scales had an AUROC of 96% (CI 0.9448 to 0.9855 at 95%, p<0.0001 and CI 0.9464 to 0.9872 at 95%, p<0.0001, respectively) (Figure 2).
The cutoff value for the COVID-HOP scale was found to be 52.7, while the COVID ICOP was 113.1 and the COVID-MOP was 109 (Supp. Fig. 2). Thus, the complete algorithms with cutoff values were designed as detailed in Table 1, where the magnitude of function of each marker, given by the Pearson’s correlation index, would add each time the patient presents pathological levels of it, and if the scale’s cut-off value is exceeded the patient would be considered at risk of either dying, needing regular hospitalization or intensive care.
Table 1. Algorithms for COVID-HOP, COVID-ICOP and COVID-MOP calculations.
Furthermore, the SE, SP, PPV, NPV (Table 2) and OR (Supp. Fig. 3) for each OP scale with its respective cutoff value were calculated, finding that the COVID-HOP had a SE of 86%, SP of 74%, PPV of 90%, NPV of 94% and OR of 18.4 (CI at 95% 8.6 to 36, p≤0.0001). On the other hand the COVID-ICOP had a SE of 87%, SP of 92%, PPV 92%, NPV of 88% and OR of 88.5 (CI at 95% 35-191, p≤0.0001). Finally, the COVID-MOP has a SE of 92%, SP of 91%, PPV of 82%, NPV of 96% and OR of 131 (CI at 95% 47-341, p≤0.0001).
Table 2
Sensitivity, specificity, positive and negative predictive values for the outcome prognostic scales.
Variable
|
Effect Size
|
CI at 95%
|
p value
|
COVID-HOP
|
|
|
Sensitivity
|
0.8629
|
0.8040 to 0.9061
|
0.0001
|
Specificity
|
0.7458
|
0.6220 to 0.8394
|
0.0001
|
Positive Predictive Value
|
0.9096
|
0.8563 to 0.9445
|
0.0001
|
Negative Predictive Value
|
0.6471
|
0.5284 to 0.7500
|
0.0001
|
COVID-ICUOP
|
|
|
Sensitivity
|
0.8710
|
0.8006 to 0.9190
|
0.0001
|
Specificity
|
0.9291
|
0.8708 to 0.9623
|
0.0001
|
Positive Predictive Value
|
0.9231
|
0.8603 to 0.9590
|
0.0001
|
Negative Predictive Value
|
0.8806
|
0.8148 to 0.9252
|
0.0001
|
COVID-MOP
|
|
|
|
Sensitivity
|
0.9241
|
0.8440 to 0.9647
|
0.0001
|
Specificity
|
0.9153
|
0.8649 to 0.9480
|
0.0001
|
Positive Predictive Value
|
0.8295
|
0.7376 to 0.8939
|
0.0001
|
Negative Predictive Value
|
0.9643
|
0.9243 to 0.9835
|
0.0001
|
Abbreviations: HOP, hospitalization-outcome prognostic; ICOP, intensive care outcome-prognostic; MOP, mortality outcome-prognostic. |
Furthermore, 167 patients’ records belonging to a different health center and that were not used to calculate the algorithms, were retrospectively studied to perform a validation of the SE, SP, PPV, NPV. Only 26% of these patients (32 individuals) were vaccinated against the coronavirus. The results for the MOP algorithm showed no variation in the second population tested, while the ICOP scale exhibited only minimal variation. In regards to the HOP algorithm, the specificity was considerably reduced (0.74 in the creation of the algorithm, 0.36 in the test of accuracy), but the other parameters remained without significant changes (Table 3).
Table 3
Validation experiment for the HOP, ICOP and MOP algorithms.
Variable
|
Effect Size
|
CI at 95%
|
p value
|
COVID-HOP
|
|
|
Sensitivity
|
0.9398
|
0.8666 to 0.9740
|
0.0001
|
Specificity
|
0.3684
|
0.2338 to 0.5272
|
0.0001
|
Positive Predictive Value
|
0.7647
|
0.6737 to 0.8365
|
0.0001
|
Negative Predictive Value
|
0.7368
|
0.5121 to 0.8819
|
0.0001
|
COVID-ICOP
|
|
|
Sensitivity
|
0.9365
|
0.8478 to 0.9750
|
0.0001
|
Specificity
|
0.8596
|
0.7468 to 0.9271
|
0.0001
|
Positive Predictive Value
|
0.8806
|
0.7817 to 0.9382
|
0.0001
|
Negative Predictive Value
|
0.9245
|
0.8214 to 0.9703
|
0.0001
|
COVID-MOP
|
|
|
|
Sensitivity
|
0.9241
|
0.8440 to 0.9647
|
0.0001
|
Specificity
|
0.9153
|
0.8649 to 0.9480
|
0.0001
|
Positive Predictive Value
|
0.8295
|
0.7376 to 0.8939
|
0.0001
|
Negative Predictive Value
|
0.9643
|
0.9243 to 0.9835
|
0.0001
|