3.1 Patient demographics
The median age of the 382 patients with COVID-19 was 39 years; 233 were male and 149 were female (Table 1). Of the 51 inpatients who required oxygen, 30 required low-flow oxygen, three needed high-flow oxygen, eight were treated with invasive ventilation/ECMO, and 10 died.
3.2 Prognostic predictability of COVID-19 at initial diagnosis
To evaluate the predictability of COVID-19 prognosis at initial diagnosis, we constructed prognostic models according to the following information available at initial diagnosis: symptoms, background information, and biomarkers. We predicted the maximum level of severity [21] that patients would reach. The AUC was 0.717 for predicting whether the severity would be ≥2, 0.878 for ≥3, 0.951 for ≥4, 0.952 for ≥5, and 0.970 for 6 (Fig. 2a). When we examined the concordance between the predicted probability and the actual severity outcome, we found that most patients with an actual severity of ≥3 had a high probability of have a severity of ≥2, but patients with an actual severity of 1 or 2 showed a wide distribution from low to high predicted probability, indicating that there is no clear distinction between them (Fig. 2b, top panel). For the other predictions, the predicted probabilities and actual severity showed good concordance; however, there were some cases where the risk of severity was not properly determined, e.g., those that ended up with a mild disease but had a high probability of being severe (Fig. 2b).
Subsequently, we evaluated the variable importance of each prediction. The presence of pneumonia was identified as the most important predictor of whether severity would be ≥2 (Fig. 2c, leftmost panel). Most patients with a severity of ≥2 who were hospitalized showed pneumonia at initial diagnosis, and all patients with a severity of ≥3 showed pneumonia (Fig. 2d). Conversely, as the target severity level of prediction increased, the importance of symptoms decreased; yet, biomarkers such as the lymphocyte count, prothrombin time (PT), and C-reactive protein (CRP), creatinine, and amylase levels were the top predictors (Fig. 2c). BMI was also an important factor in predicting whether the severity was ≥4 or ≥5. Age was an important factor for all severity levels. Among the important predictors, we examined the distribution of age and biomarkers according to the actual severity prognosis and found that age and BMI were higher in patients with a severity of ≥3 than in others, and biomarkers such as PT, CRP, creatinine, red blood cell volume distribution width, and blood glucose values were higher in patients with a severity ≥4 than in others (Fig. 2e). Lymphocyte counts, estimated glomerular filtration rate (eGFR)-creatinine, and albumin values were lower in more severe cases than in less severe cases, and platelet counts were specifically low in mortality cases (Fig. 2e). The amylase level showed a peculiar distribution, being low in patients with severity levels of 4, 5, and 6 and very high in some patients with a severity level of 6 (Fig. 2e).
3.3 Dynamic mortality risk assessment based on longitudinal data during hospitalization
Outcome screening based on information from the initial diagnosis was accurate but incomplete. Therefore, the patient's prognosis is not entirely determined at the initial diagnosis, and there is room for change in prognosis depending on the disease course and treatment after admission. When we investigated the changes in severity status after admission, we found that oxygen administration and noninvasive ventilation were initiated within 5 days after admission, whereas invasive ventilation was often introduced >5 days after admission; death occurred >20 days after admission (Fig. 3a). Additionally, many deaths occurred without invasive ventilation (Fig. 3a, top panel). This is because most mortality cases involved elderly individuals who were not eligible for invasive ventilation or ECMO even when their condition deteriorated. These observations suggest that during the few weeks between admission and death or discharge, patients undergo changes in their condition that cannot be fully assessed based on their oxygen-support status.
Then, we used RSF [17] to evaluate the mortality risk of the patients during hospitalization. For all four mortality cases in the validation dataset, an increase in mortality risk was observed approximately 1–2 weeks before the outcome, and cumulative hazard function (CHF; i.e., the estimated probability of death within 7 days) reached approximately >0.3 at the time of death (Fig. 3b). Conversely, in patients recovering from invasive or noninvasive ventilation, the CHF increased at approximately 1 week after admission, as in the mortality cases, but then decreased, and the CHF rarely exceeded 0.2. In mild cases that progressed with oxygen administration or room air, there was little increase in CHF. Similar changes in CHF were observed in the training dataset, which included six deaths, suggesting the generality of the predictive model between the training and validation datasets (Fig. 3c).
The average contribution of the variables to the RSF prediction for each patient was then evaluated using the mean aggregated SurvSHAP(t). In addition to background factors, such as age and BMI, blood test items that were measured multiple times during hospitalization, such as platelets, amylase, and β-D-glucan, had the highest importance in mortality prediction (Fig. 3d). Age, amylase level, and platelet count were identified as important predictors of mortality at initial diagnosis. Most of the important variables showed high contributions only in mortality cases; however, some, such as age and BMI, showed a high mean aggregated SurvSHAP(t) in milder cases, suggesting that they are nonspecific factors. Conversely, important blood test items such as β-D-glucan, platelets, and calcium (Ca) also contributed less in some mortality cases, suggesting heterogeneity of deterioration. In the training data, these predictors were also of high contribution, especially in mortality cases, although there was some shuffling of the rankings (Fig. 3e).
3.4 Explanation of the rationale for estimated mortality risk in severe cases
For the mortality risk assessment of COVID-19, a machine-learning model based on blood markers was proposed in Nature Machine Intelligence (NMI) 2020 [25]. This model was built using samples immediately prior to death or discharge, and lactate dehydrogenase (LDH), lymphocytes, and CRP were identified as key features. Mortality risk assessment using a decision tree based on these three features has been proposed in the study to facilitate clinical application (Fig. 4a). After about 10 days of hospitalization, the RSF and NMI models showed equivalent performance as measured by accuracy and F1-score. However, immediately following hospitalization, the RSF model outperformed the NMI model, indicating its superiority in early prognostic prediction (Fig. 4b and 4c).
To examine the factors contributing to mortality risk over time, we calculated SurvSHAP(t) daily for the severe cases included in the validation dataset. The patients and times had different combinations of factors associated with CHF changes (Fig. 4d). For example, in young patients #2 and #6, saturation of peripheral oxygen (SpO2) was associated with an increase in CHF immediately after admission. However, the contribution of SpO2 decreased with the application of ECMO/invasive ventilation. In contrast, in patients #3 and #4, who were elderly and not eligible for ECMO/invasive ventilation, the contribution of SpO2 was higher >10 days after hospitalization. β-D-glucan had a higher contribution 1–2 weeks before death than at any other time in patients #3, #4, and #6. Platelets showed a high contribution in patients #4 and #6, and BMI also contributed to the mortality risk in these patients. Of note, in patient #4, BMI was measured for the first time on day 7, so there was a rapid increase in the contribution of BMI on day 7. Ca and blood amylase levels were elevated immediately before death, suggesting electrolyte abnormalities and multi-organ failure. CRP, which is also a major predictor of mortality at initial diagnosis, contributed immediately after hospitalization.