Machine learning-based prediction model for volume responsiveness in critically ill patients with oliguric acute kidney injury

doi:10.21203/rs.3.rs-3209707/v1

Download PDF

Research Article

Machine learning-based prediction model for volume responsiveness in critically ill patients with oliguric acute kidney injury

https://doi.org/10.21203/rs.3.rs-3209707/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background and objectives:

Fluid balance in acute kidney injury (AKI) patients can have adverse consequences if it is too high or too low, so rational fluid management is needed according to the patient’s volume status. This study aimed to develop a prediction model that can effectively identify volume-responsive (VR) and volume-unresponsive (VU) AKI patients.

Methods

We selected AKI patients from the US-based critical care database (Medical Information Mart for Intensive Care, MIMIC-IV2.2) who had urine output <0.5 ml/kg/h in the first 6 h after ICU admission and fluid intake >5 l in the next 6 h. Patients who received diuretics and renal replacement therapy on day 1 were excluded. We developed three predictive models, based on either machine learning Gradient Boosting Machine (GBM), random forest or logistic regression, to predict urine output >0.65 ml/kg/h in the 18 h following the initial 6 h of oliguria assessment, we divided the whole sample into training and testing sets by a ratio of 3:1，after training and optimizing the model, ranked the importance of features and evaluated the stability and accuracy of the model.

Main results

We analyzed 6295 patients, of whom 1438 (22.8%) experienced volume responsiveness and exhibited increased urine output after receiving more than 5 liters of fluid. Urinary creatinine, blood urea nitrogen (BUN), blood glucose and age were identified as important predictive factors for volume responsiveness. The Random Forest model performed the best, followed by the GBM model.The machine learning GBM outperformed the traditional logistic regression model in distinguishing between the volume responsive (VR) and volume unresponsive (VU) groups (AU-ROC, 0.874; 95% CI, 0.867 to 0.874 vs. 0.789; 95% CI, 0.779 to 0.789, respectively).

Conclusions

The Random Forest and GBM model, compared to the traditional logistic regression model, demonstrated a better ability to differentiate patients who did not exhibit a response in urine output to fluid intake. This finding suggests that machine learning techniques have the potential to improve the development and validation of predictive models in critical care research. Based on the feature importance ranking, creatinine, bun, age, glucose, and bicarbonate were identified as highly important features in the model could predicted VR in AKI patients.

AKI patients

Volume responsiveness

Machine learning

Random Forest

GBM

Acute Kidney Injury (AKI) is a common syndrome of acute renal function impairment in critically ill patients¹. Numerous studies have demonstrated that even a minimal elevation in serum creatinine can significantly increase the risk of mortality^2,3. According to the KDIGO guidelines, AKI can be defined as an increase in serum creatinine or a decrease in urine output ⁴. In the overall population of AKI, oliguric AKI is common and presents a challenge for fluid management. Oliguria may be an adaptive response in AKI, and once circulation volume is restored, urine output improves, and fluid balance can be considered beneficial. Patients who show improved urine output may have volume-responsive (VR) AKI.

However, it is important to note that even in patients who respond to fluids, sustained improvement in tissue perfusion may not always be achieved through intravenous fluid administration⁵. Additionally, patients with sepsis and distributive shock may show signs of fluid responsiveness while simultaneously experiencing adverse effects⁶. The assessment of potential harm caused by fluid administration is known as fluid tolerance evaluation ⁷. AKI patients, it is crucial to first evaluate fluid tolerance before assessing fluid responsiveness, it is important to distinguish between VU and VR in AKI patients.

The accurate identification of volume responsiveness in critically ill patients with AKI is crucial in clinical practice to prevent both fluid overload and hypovolemia⁸. Currently, there is a lack of reliable targets for distinguishing between VR and VU AKI during the early stages^9–12.

Although the passive leg raising (PLR) maneuver and fluid challenge are reliable techniques for assessing fluid responsiveness^13–16, both the PLR maneuver and fluid challenge have limitations, for instance, excessive fluid administration during a fluid challenge can cause fluid overload, especially in patients with impaired cardiac, renal, or pulmonary function. This can lead to vascular overfilling and tissue edema, among other complications.

In our study, we examine the metrics related to fluid responsiveness using modeling analysis techniques such as GBM (Gradient Boosting Machine) and Random Forest. These machine learning approaches allow us to delve deeper into the interactions among various metrics and gain better management of VU and VR AKI, benefiting patient care.

Source of data

We used a large US-based critical care database named Medical Information Mart for Intensive Care (MIMIC-IV 2.2) for our analysis. MIMIC-IV is an integrated, deidentified, comprehensive clinical dataset that covers all patients who visited the intensive care units (ICUs) of Beth Israel Deaconess Medical Center in Boston, MA, from 2008 to 201.17. The latest data from the database consists of 299,712 patients, with a total of 431,231 admissions and 73,181 ICU stays, these numbers reflect the most up-to-date information available and provide valuable insights for analysis and research purposes. As this study was an analysis of a third-party anonymized publicly available database with pre-existing institutional review board (IRB) approval, IRB approval from our institution was not required. The study was reported following the recommendations of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statements.

Participants

Patients were considered eligible if their urine output remained below 0.5 ml/kg/h for the first 6 hours after admission to the ICU. This criterion aligned with the urine output component of the KDIGO criteria4. To assess the impact of fluid resuscitation on subsequent urine output, only patients who received a substantial fluid intake (> 5 l) within 6 hours after the initial 6-hour period of oliguria assessment were included (Fig. 1). Fluid intake and urine output data were extracted from the nursing chart system, providing a more accurate reflection of actual volume intake or output compared to the medical order system. Patients who were discharged from the ICU during the observation period were excluded. Additionally, patients receiving diuretics and/or renal replacement therapy (RRT) on the first day were also excluded.

Outcome (volume responsiveness)

The urine output within 18 hours after the initial 6 hours for defining oliguria was used as the outcome measure. Patients were classified as VR-AKI if their urine output was greater than 0.65 ml/kg/h, representing a 30% increase compared to the baseline value. Conversely, if their urine output did not meet these criteria, were classified as VU-AKI.

Predictors of VR-AKI

Routinely collected clinical and laboratory variables obtained within the first 6 h of ICU admission were assessed for their ability to predict volume responsiveness. For some variables with multiple measurements, both the maximum and minimum values were assessed. Age, gender, ethnicity, type of ICU, length of hospital stay and ICU stay, patient survival status, and vital signs including respiratory rate, blood pressure, heart rate, and temperature were analyzed. In addition, laboratory data including glucose, white blood cell count (WBC), platelet count, hematocrit, Hemoglobin, diastolic blood pressure, chloride, potassium, sodium, Calcium, bicarbonate, lactate, creatinine, blood urea nitrogen (BUN), coagulation profile, and SpO₂ were included.

In this preliminary epidemiological study, the sample size was not predetermined. Instead, all eligible patients in the database were included to maximize the statistical power of the predictive model. However, certain variables such as albumin, AST, ALT, amylase, and bilirubin, which had missing values exceeding 60%, were not considered for further analysis. All other variables had less than 20% missing values, multiple imputation methods and median imputation were employed to handle missing values¹⁸. For more details, please refer to supplemental table S1 and table S2, which provide comprehensive information on the patient dataset.

Model development

In this study, the Gradient Boosting Machine (GBM) method was employed to predict VR versus VU. A set of model parameters, including interaction depth, number of trees, shrinkage, and minimum observations in a leaf node, was specified. The optimal parameter combination was determined through a grid search within the given parameter ranges.

The GBM model was trained using the training dataset, with VR as the response variable and other features as predictors. Cross-validation was used during the training process to evaluate the model's performance and ensure its generalization ability.

After training the model, the classification performance was evaluated by calculating the Receiver Operating Characteristic (ROC) curve and the Area Under the Curve (AUC) using the test dataset. To see the results and performance metrics, please refer to supplemental table S3. The ROC curve, which visually represents the model’s predictive accuracy, can be seen in supplemental table S4 as well. The plot function was used to visualize the ROC curve, where a higher AUC value indicates better predictive accuracy.

The uncertainty of the AUC was assessed using confidence intervals (CI), calculated using the bootstrap method. The CI values were displayed on the ROC curve plot. Additionally, feature importance was extracted from the model, ensuring the accurate assessment of feature importance based on the optimal number of trees. We also fitted several other models including logistic regression and random forest for comparison. The AU-PRC(area under the precision-recall curve, AU-PRC), AU-ROC(area under the receiver operating characteristic curve accuracy, AU-ROC), and BS (BS, Brier score) were compared with the GBM model, and further details regarding the methodology can be found in supplemental table S5.

Statistical analysis

The data analysis was conducted using R version 3.5.0 (RStudio). For continuous variables, the results were presented as either the median (interquartile range), depending on the distribution of the data. Categorical variables were reported as counts and percentages.To assess the differences between continuous and categorical parameters, statistical tests such as the Wilcoxon rank sum test, chi-square test, or Fisher exact test were performed, as appropriate. The Delong test was utilized to compare different areas under the curve (AUCs).

The odds ratio (OR) for the difference in survival between the responsive and unresponsive groups was calculated using logistic regression. To identify variables that were predictive of volume responsiveness, a stepwise logistic regression model was utilized. This model employed backward elimination methods, testing at each step to determine which variables should be excluded. All statistical tests were two-sided and conducted at a significance threshold of 0.05.

The major packages of R employed in this study are listed in Table S6. The code for accessing the MIMIC database can be obtained from https://github.com/MIT-LCP/mimic-code/.

Participants

Among the 23992 patients with urine output < 0.5 ml/kg/h for the first 6 h after ICU admission, a total of 7808 patients (32.5%) received fluid intake > 5 l within the subsequent 6 h. Additionally, 1511patients were excluded because they received diuretics and/or RRT on the first day. Therefore, our analysis included a total of 6295 patients, out of which 1438 patients had VR-AKI, and 4856 patients had VU-AKI on day 1 in the ICU (Fig. 2).

The differences in characteristics between the VR and VU groups are described in Table 1. The VR group had a higher maximum serum creatinine concentration compared to the VU group (1.5 [IQR 1, 2.4] vs. 1 [IQR 0.8, 1.3]; p < 0.001). Similarly, the VR group had a higher maximum blood urea nitrogen (BUN) (29 [IQR 19, 45] vs. 18 [IQR 14, 26]; p < 0.001) as well as a higher minimum creatinine level (1.1 [IQR 0.8, 1.9] vs. 0.8 [IQR 0.6, 1]; p < 0.001) and a higher minimum BUN (23 [IQR 15, 38] vs. 15 [IQR 11, 21]; p < 0.001). The VR group also had a higher age (69.61 [IQR 58.8, 79.88] vs. 67.27 [IQR 57.41, 76.47]; p < 0.001) and longer length of stay in the ICU (4.12 [IQR 2.24, 7.41] vs. 3.31 [IQR 1.85, 6.16]; p < 0.001).

Regarding respiratory parameters, the VR group had a higher mean respiratory rate (19.26 [IQR 16.87, 22.29] vs. 18.21 [IQR 16.36, 21.12]; p < 0.001), additionally, the VR group had a higher mean oxygen saturation (97.46 [IQR 95.92, 98.74] vs. 97.71 [IQR 96.36, 98.8]; p < 0.001).

In terms of other laboratory values, the VR group had a higher maximum glucose level (176 [IQR 139, 228] vs. 174 [IQR 147, 210]; p = 0.254), a lower minimum bicarbonate concentration (21 [IQR 17, 23] vs. 22 [IQR 20, 24]; p < 0.001), and a lower maximum platelet count (207 [IQR 146, 285] vs. 201 [IQR 151, 262.5]; p = 0.186). The VR group also had a higher mean systolic blood pressure (109.33 [IQR 101.82, 118.83] vs. 110.91 [IQR 104.7, 120.45]; p < 0.001) and a higher mean heart rate (87.33 [IQR 76.64, 100.12] vs. 83.88 [IQR 75.63, 94.46]; p < 0.001). (Table 1).

Table 1

Patient characteristics and clinical variables.
Variables	Total (n = 6295)	0 (n = 4857)	1 (n = 1438)	p
demographic
Age, Median (Q1,Q3)	69.03 (58.45, 79.23)	69.61 (58.8, 79.88)	67.27 (57.41, 76.47)	< 0.001
gender, n (%)				0.003
0	2695 (43)	2129 (44)	566 (39)
1	3600 (57)	2728 (56)	872 (61)
Ethnicity (%)
ASIAN	126 (2.0)	97 (2.0)	29 (2.0)	0.317
BLACK	582 (9.0)	478(9.8)	104 (7.2)
HISPANIC OR LATINO	169 (2.8)	124 (2.5)	45 (3.1)
MULTIPLE RACE/ETHNICITY	7 (0.1)	7 (0.1)	0 (0.0)
NATIVE HAWAIIAN OR OTHER PACIFIC ISLANDER	16 (0.3)	14 (0.3)	2 (0.1)
PATIENT DECLINED TO ANSWER	39 (0.6)	27 (0.6)	12 (0.8)
PORTUGUESE	31 (0.5)	28 (0.6)	3 (0.2)
SOUTH AMERICAN	3 (0.1)	2 (0.00)	1 (0.1)
UNKNOWN	951 (15.5)	720 (14.8)	231 (16.1)
WHITE	4371 (71.3)	3360 (69.2)	1011 (70.3)
ICU type (%)
Cardiac Vascular Intensive Care Unit (CVICU)	1467 (23.3)	866 (17.8)	601 (41.8)	< 0.001
Coronary Care Unit (CCU)	514 (8.6)	384 (7.9)	130 (9.0)
Medical Intensive Care Unit (MICU)	1476 (23.4)	1254 (25.8)	222 (15.4)
Medical/Surgical Intensive Care Unit (MICU/SICU)	978 (15.7)	818 (16.8)	160 (11.1)
Neuro Intermediate	13 (0.2)	10 (0.2)	3 (0.2)
Neuro Stepdown	13 (0.2)	6 (0.1)	5 (0.3)
Neuro Surgical Intensive Care Unit (Neuro SICU)	80 (1.3)	55 (1.1)	25 (1.7)
Surgical Intensive Care Unit (SICU)	(14.2)	730 (15.0)	132 (9.2)
Trauma SICU (TSICU)	(14.7)	734 (15.1)	160 (11.1)
Mortality Status, n (%)				< 0.001
No	3122 (50)	2160 (44)	962 (67)
Yes	3173 (50)	2697 (56)	476 (33)
ICU LOS (days)	3.94 (2.14, 7.15)	4.12 (2.24, 7.42)	3.31 (1.85, 6.16)	< 0.001
Hospital LOS (days)	10.24 (6.21, 17.65)	10.86 (6.43, 18.72)	8.85 (5.77, 14.52)	< 0.001
Weight (kg)	87.3 (72.85, 103.85)	88 (73, 105.3)	85.8 (72.4, 99)	< 0.001
Urine Output at 6 hours (ml)	160 (100, 215)	150 (80, 205)	195 (150, 238)	< 0.001
Urine Output at 12 hours (ml)	350 (210, 498.5)	305 (165, 433)	535 (395, 779.5)	< 0.001
Urine Output at 24 hours (ml)	827 (469, 1311)	670 (375, 975)	1805 (1430, 2279.5)	< 0.001
Urine Output Trend at 6 hours (ml/h)	5 (4, 5)	5 (4, 5)	5 (5, 5)	< 0.001
Urine Output Trend at 12 hours (ml/h)	11 (10, 11)	11 (10, 11)	11 (10.69, 11)	< 0.001
Urine Output Trend at 24 hours (ml/h)	22.68 (21.57, 23)	22.5 (21.22, 23)	23 (22, 23)	< 0.001
Minimum Heart Rate (bpm)	71 (61, 83)	72 (61, 84)	69 (60, 79)	< 0.001
Maximum Heart Rate (bpm)	105 (91, 121)	106 (92, 122)	102 (90, 117)	< 0.001
Mean Heart Rate (bpm)	86.56 (76.39, 98.96)	87.34 (76.64, 100.12)	83.88 (75.63, 94.46)	< 0.001
Minimum Systolic Blood Pressure (mmHg)	84 (75, 92)	83 (74, 92)	86 (78, 94)	< 0.001
Maximum Systolic Blood Pressure (mmHg)	142 (129, 158)	142 (129, 158)	143 (131, 158)	0.046
Mean Systolic Blood Pressure (mmHg)	109.8 (102.62, 119.11)	109.33 (101.82, 118.83)	110.91 (104.7, 120.45)	< 0.001
Minimum Diastolic Blood Pressure (mmHg)	43 (37, 49)	42 (36, 49)	44 (39, 50)	< 0.001
Maximum Diastolic Blood Pressure (mmHg)	82 (71, 96)	82 (71, 96)	82 (72, 94)	0.259
Mean Diastolic Blood Pressure (mmHg)	58.44 (52.79, 65.09)	58.26 (52.4, 64.83)	59.3 (54.19, 65.84)	< 0.001
Minimum Mean Blood Pressure (mmHg)	55 (48, 61)	55 (47, 61)	57 (51, 63)	< 0.001
Maximum Mean Blood Pressure (mmHg)	99 (89, 113)	99 (88, 113)	99 (90, 112)	0.169
Mean Mean Blood Pressure (mmHg)	73.44 (68.1, 79.6)	72.92 (67.59, 79.1)	74.74 (70.05, 80.86)	< 0.001
Minimum Respiratory Rate (bpm)	12 (10, 15)	12 (10, 15)	12 (9, 14)	< 0.001
Maximum Respiratory Rate (bpm)	28 (24, 32)	28 (24, 32)	27 (24, 32)	0.01
Mean Respiratory Rate (bpm)	19 (16.73, 22.04)	19.26 (16.88, 22.29)	18.21 (16.36, 21.12)	< 0.001
Minimum Temperature (°C)	36.39 (35.83, 36.61)	36.39 (35.83, 36.67)	36.39 (35.83, 36.61)	0.615
Maximum Temperature (°C)	37.28 (36.94, 37.83)	37.28 (36.94, 37.83)	37.33 (37, 37.89)	0.004
Mean Temperature (°C)	36.8 (36.52, 37.13)	36.8 (36.5, 37.13)	36.82 (36.55, 37.16)	0.024
Minimum SpO2 (%)	93 (90, 95)	93 (90, 95)	93 (90, 95)	< 0.001
Maximum SpO2 (%)	100 (100, 100)	100 (100, 100)	100 (100, 100)	< 0.001
Mean SpO2 (%)	97.54 (96, 98.76)	97.46 (95.92, 98.74)	97.71 (96.36, 98.8)	< 0.001
Minimum Glucose (mg/dl)	102 (85, 126)	104 (85, 128)	97 (83, 117)	< 0.001
Maximum Glucose (mg/dl)	176 (141, 223)	176 (139, 228)	174 (147, 210)	0.251
Mean Glucose (mg/dl)	135.33 (117.33, 165)	137.17 (116.5, 169.25)	130.94 (118.89, 152.04)	< 0.001
Minimum Hematocrit (%)	28.2 (24.2, 32.9)	28.1 (24.1, 32.95)	28.4 (24.75, 32.6)	0.056
Maximum Hematocrit (%)	33.6 (30, 38.2)	33.4 (29.8, 38)	34.4 (30.6, 38.6)	< 0.001
Minimum Hemoglobin (g/dl)	9.3 (8, 10.8)	9.2 (7.9, 10.8)	9.5 (8.2, 11)	< 0.001
Maximum Hemoglobin (g/dl)	11 (9.7, 12.5)	10.9 (9.6, 12.4)	11.4 (10, 12.9)	< 0.001
Minimum Platelets (cells/nL)	158 (106, 224)	159 (104, 227)	155 (113, 212.5)	0.698
Maximum Platelets (cells/nL)	205 (147, 279)	207 (146, 285)	201 (151, 262.5)	0.186
Minimum WBC (cells/nL)	10.15 (7.1, 13.9)	10.3 (7, 14.4)	9.7 (7.1, 12.8)	< 0.001
Maximum WBC (cells/nL)	14.7 (10.6, 20.1)	14.9 (10.6, 20.6)	14.2 (10.6, 18.85)	< 0.001
Minimum Anion Gap (mmol/L)	13 (11, 15)	13 (11, 16)	12 (10, 14)	< 0.001
Maximum Anion Gap (mmol/L)	16 (13, 20)	17 (14, 20)	14 (12, 17)	< 0.001
Minimum Bicarbonate (mmol/L)	21 (18, 24)	21 (17, 23)	22 (20, 24)	< 0.001
Maximum Bicarbonate (mmol/L)	24 (21, 26)	23 (21, 26)	25 (23, 27)	< 0.001
Minimum BUN (mg/dl)	21 (14, 34.75)	23 (15, 38)	15 (11, 21)	< 0.001
Maximum BUN (mg/dl)	25 (17, 41)	29 (19, 45)	18 (14, 26)	< 0.001
Minimum Calcium (mg/dl)	7.9 (7.4, 8.4)	7.8 (7.3, 8.4)	8.1 (7.6, 8.5)	< 0.001
Maximum Calcium (mg/dl)	8.4 (8, 9)	8.4 (7.9, 8.9)	8.5 (8.1, 9)	< 0.001
Minimum Chloride (mmol/L)	103 (99, 106)	103 (98, 106)	104 (100, 107)	< 0.001
Maximum Chloride (mmol/L)	107 (103, 110)	106 (102, 110)	108 (104, 111)	< 0.001
Minimum Creatinine (mg/dl)	1 (0.7, 1.7)	1.1 (0.8, 1.9)	0.8 (0.6, 1)	< 0.001
Maximum Creatinine (mg/dl)	1.3 (0.9, 2.1)	1.5 (1, 2.4)	1 (0.8, 1.3)	< 0.001
Minimum Glucose (mg/dl)	114 (95, 138)	114 (94, 140)	114 (98, 134)	0.788
Maximum Glucose (mg/dl)	150 (121, 202)	156 (123, 208)	140 (116, 178)	< 0.001
Minimum Sodium (mmol/L)	137 (134, 140)	137 (134, 140)	137 (135, 139)	0.003
Maximum Sodium (mmol/L)	140 (137, 142)	140 (137, 142)	140 (138, 142)	0.021
Minimum Potassium (mmol/L)	4 (3.6, 4.4)	4 (3.6, 4.4)	3.9 (3.6, 4.2)	< 0.001
Maximum Potassium (mmol/L)	4.6 (4.2, 5.1)	4.6 (4.2, 5.2)	4.5 (4.1, 4.8)	< 0.001
Minimum INR	1.3 (1.1, 1.5)	1.3 (1.1, 1.6)	1.2 (1.1, 1.3)	< 0.001
Maximum INR	1.4 (1.2, 1.8)	1.5 (1.2, 2)	1.4 (1.2, 1.6)	< 0.001
Minimum PT	13.9 (12.5, 16.4)	14.1 (12.6, 17)	13.3 (12.2, 14.8)	< 0.001
Maximum PT	15.8 (13.7, 20)	16.1 (13.7, 21.2)	15.2 (13.4, 17.3)	< 0.001
Minimum PTT	29.7 (26.4, 35.4)	30.3 (26.6, 36.4)	28.35 (25.9, 32.6)	< 0.001
Maximum PTT	35.5 (29.7, 51.6)	36.4 (29.9, 54.8)	33.3 (28.9, 42.98)	< 0.001

The stepwise logistic regression model

The results of the stepwise logistic regression model are shown in Table 2.

Table 2

Presents the results of the multivariable logistic regression model with stepwise variable selection.
Variable	OR	Lower_CI	Upper_CI	P value
Maximum Chloride (mmol/L)	1.799223	1.664057	1.945368	<0.001
Minimum Calcium (mg/dL)	1.458282	1.381185	1.539683	<0.001
Minimum Bicarbonate (mmol/L)	1.369065	1.266411	1.480041	<0.001
Mean Mean Blood Pressure (mmHg)	1.25605	1.133137	1.392295	<0.001
Gender	1.246044	1.203159	1.290457	<0.001
Maximum Respiratory Rate (breaths/min)	1.185653	1.133806	1.239871	<0.001
Maximum Hemoglobin (g/dL)	1.174004	1.10788	1.244074	<0.001
Minimum SpO2 (%)	1.110225	1.051461	1.172273	0.0002
Minimum Platelets (10^3 cells/µL)	1.107218	0.996011	1.230843	0.0593
Maximum Temperature (°C)	1.098948	1.057821	1.141673	<0.001
Length of Stay in ICU (hours)	1.079387	1.011251	1.152114	0.0217
Maximum WBC (10^3 cells/µL)	1.071927	0.984359	1.167285	0.1102
Minimum PTT (seconds)	1.066518	1.025768	1.108886	0.0012
Mean Respiratory Rate (breaths/min)	1.065308	1.003116	1.131355	0.0393
Minimum Anion Gap (mmol/L)	1.06476	0.9995	1.134281	0.0518
Minimum Glucose (mg/dL)	1.062127	1.003791	1.123852	0.0365
Mean SBP (mmHg)	1.055269	0.988131	1.126968	0.1087
Minimum Respiratory Rate (breaths/min)	0.941303	0.899962	0.984544	0.0083
Maximum BUN (mg/dL)	0.938812	0.874558	1.007787	0.0809
Mean SpO2 (%)	0.932862	0.885043	0.983265	0.0096
Maximum PT (seconds)	0.932426	0.876491	0.991931	0.0266
Maximum SBP (mmHg)	0.930431	0.886336	0.976719	0.0036
Maximum Anion Gap (mmol/L)	0.915221	0.842399	0.994339	0.0362
Minimum Temperature (°C)	0.914751	0.880083	0.950785	0.0000
Minimum Sodium (mmol/L)	0.914647	0.850887	0.983184	0.0155
Minimum WBC (10^3 cells/µL)	0.903147	0.829736	0.983052	0.0185
Minimum Heart Rate (bpm)	0.88956	0.855544	0.924928	0.0000
Mean DBP (mmHg)	0.872301	0.801546	0.949302	0.0015
Maximum Glucose (mg/dL)	0.872201	0.82218	0.925264	<0.001
Minimum Potassium (mmol/L)	0.866115	0.832321	0.90128	<0.001
Minimum INR	0.860347	0.806006	0.918351	<0.001
Maximum Platelets (10^3 cells/µL)	0.848704	0.766685	0.939497	0.0016
Fluid Amount in 6 hours (mL)	0.83502	0.780074	0.893835	<0.001
Minimum Hematocrit (%)	0.833272	0.784135	0.885488	<0.001
Maximum Calcium (mg/dL)	0.828728	0.785335	0.874518	<0.001
Length of Stay in Hospital (days)	0.818164	0.778463	0.85989	<0.001
Admission Age (years)	0.792257	0.76291	0.822732	<0.001
Death	0.77022	0.741129	0.800454	<0.001
Minimum Chloride (mmol/L)	0.618286	0.563874	0.677947	<0.001
Weight (kg)	0.608458	0.580695	0.637548	<0.001
Maximum Creatinine (mg/dL)	0.262808	0.232765	0.296728	<0.001

When the odds ratio (OR) value is greater than 1, suggests that the presence of a variable or an increase in a continuous variable is linked to an increased likelihood of volume responsiveness.

Abbreviations: ICU Intensive Care Unit， bpm Beats per minute， WBC White Blood Cells， SBP Systolic Blood Pressure， BUN Blood Urea Nitrogen， SpO2 Oxygen Saturation， PT Prothrombin Time， DBP Diastolic Blood Pressure， INR International Normalized Ratio

The logistic regression analysis conducted in this study demonstrated significant associations between several factors and the incidence of Acute Kidney Injury (AKI) as well as patient outcomes. Age, for example, exhibited an odds ratio (OR) of 0.79(95% confidence interval [CI]: 0.76-0.82), indicating that older age is associated with a decreased likelihood of developing AKI.

The results of the stepwise logistic regression model, displayed in Table 2, align with our expectations. Advanced age (OR for a 20-year increase: 0.79; 95% CI: 0.77 to 0.82), higher serum creatinine levels (OR for each 0.1 mg/dl increase: 0.26; 95% CI: 0.23 to 0.30), and maximum systolic blood pressure (OR for each 20-mmHg increase: 0.93; 95% CI: 0.89 to 0.98) were all associated with a decreased probability of volume responsiveness.

Conversely, higher values of temperature (OR for each 0.1°C increase: 1.10; 95% CI: 1.06 to 1.14), mean blood pressure (OR for each 20-mmHg increase: 1.26; 95% CI: 1.13 to 1.39), and minimum bicarbonate levels (OR for each 5-mmol/L increase: 1.37; 95% CI: 1.27 to 1.48) were associated with an increased probability of volume responsiveness (Table 2).

The GBM model

In this study, a gradient boosting machine (GBM) model was employed to analyze the data. The model was trained using a 5-fold cross-validation method and predictions were saved for the final model. The model was configured with parameters through a grid search approach. These parameters included the interaction depth (options: 1, 3, and 5), the number of trees (options: 50, 100, and 150), the shrinkage rate (options: 0.01 and 0.1), and the minimum number of observations in each node (options: 10 and 20). The target variable VR was predicted using all remaining variables in the train data dataset.

Additionally, a Random Forest model was also used to analyze the data. The model was trained using a 5-fold cross-validation method and predictions were saved for the final model. Similar to the GBM model, a grid search approach was applied to optimize the model by tuning various parameters. These parameters included the interaction depth (options: 1, 3, and 5), the number of trees (options: 50, 100, and 150), the shrinkage rate (options: 0.01 and 0.1), and the minimum number of observations in each node (options: 10 and 20). The target variable VR was predicted using all remaining variables in the train data dataset. Additionally, feature importance analysis was conducted to assess the contribution of different features to the model. Overall, this study utilized a Random Forest model with customized parameter settings, along with cross-validation and feature importance analysis, to analyze the data and predict the target variable VR.

After training the model, we extracted the feature importance values, which indicate the contribution of each feature toward the overall prediction accuracy. Based on the feature importance ranking, we observed that creatinine, bun, age, glucose, and bicarbonate were identified as highly important features in the model (Fig3A.B).

Model performance

During the assessment of model performance in our study, we used several metrics, including AU-PRC, AU-ROC, and Brier Score. The GBM model exhibited the highest performance in our analysis, with an AU-ROC of 0.87(CI: 0.87-0.88) and a Brier Score of 0.26. These results indicate the GBM model's strong accuracy and discriminative ability in identifying VR and VU statuses. On the other hand, the Random Forest model also demonstrated impressive performance, with a low Brier Score of 0.03. These values highlight the model's high accuracy and discriminative ability. In contrast, the Stepwise Logistic Regression model showed relatively poorer performance, as evidenced by an AU-PRC of 0.52, AU-ROC of 0.79, and a higher Brier Score of 0.28(Table 3). The GBM had a significantly greater AU-ROC than the logistic regression model (AU-ROC, 0.874; 95% CI, 0.87 to 0.88 vs. 0.79; 95% CI, 0.78 to 0.79, respectively; Fig. 4).

Table 3

Performance Comparison of Three Statistical Analysis Methods*
Model	BS	AUROC	AUPRC
Forward Stepwise LR	0.2838692	0.7889424	0.519997
Random forest	0.02717105	0.9999716	0.9999054
GBM	0.2550571	0.8742173	0.6943073
*AUPRC, area under the precision- recall curve; AUROC, area under the receiver operating characteristic curve; BS, Brier score; e- net, elastic net; GBM, gradient boosting machine; LASSO, least absolute shrinkage and selection operator; LR, logistic regression.

The definition of AKI depends on SCr and UO, which are imperfect markers of AKI^19-21, The evaluation of urinary output (UO) is challenging without the use of a urinary catheter and requires monitoring of the patient's fluid volume and blood circulation, as well as considering the administration of diuretics. Additionally, it is a time-consuming process to assess UO on an hourly basis^19-23. Therefore, extensive research has been dedicated to the exploration and advancement of novel biomarkers.

In this study, we demonstrated that advanced machine learning techniques such as GBM (Gradient Boosting Machine) and Random Forest modeling can enhance the amount of information extracted from analyzing a database. Moreover, these techniques allow us to develop and validate predictive models that outperform traditional logistic regression methods. In our study, we demonstrated that certain clinical factors are more likely to be associated with VR-AKI compared to VU-AKI. By utilizing advanced machine learning techniques, we were able to identify several important clinical factors that are linked to VR-AKI.

Our findings revealed several relevant parameters. Firstly, the evaluation of blood urea nitrogen (BUN) and creatinine levels provides a rigorous and comprehensive approach to assess the oliguric type of acute kidney injury (AKI). Elevated BUN levels indicate impaired kidney function, while increased creatinine levels reflect diminished glomerular filtration rate^24,25. Monitoring BUN and creatinine allows for early detection and evaluation of oliguric AKI, facilitating prompt intervention. This combined assessment of BUN and creatinine yields clinically significant information, enabling accurate diagnosis and therapeutic decision-making in oliguric AKI patients, with potential implications for improving patient outcomes. Moreover, our study recognized the significance of blood glucose levels and platelet counts in assessing VR in AKI patients. Elevated blood glucose (glucose_max) that commonly occurs in diabetes and metabolic disorder patients demonstrated a potential influence on VR via various mechanisms, such as the development of hyperosmolar tubular necrosis⁶. Additionally, the minimum platelet count (platelets_min) was identified as a significant parameter, as it reflects the coagulation function and bleeding risk potential. Considering that bleeding can result in reduced fluid volume and subsequent impairment of renal perfusion, platelet count becomes an important factor in VR evaluation.

Secondly, our analysis of the random forest and GBM modeling indicated that prolonged hospital stay and extended ICU stay are crucial variables for evaluating AKI patients with an oliguric phenotype. Longer hospital stays may indicate a more complex condition, delayed recovery, and potential complications. Similarly, lengthier ICU stays suggest increased severity, organ dysfunction, and the need for intensive management. These factors reflect the disease burden, patient instability, and the challenge of achieving optimal control, making them valuable predictors in AKI evaluation²⁶.

Thirdly, the average respiratory rate (resp_rate_mean) was identified as a potential indicator of VR due to its link with respiratory function and lung health, which can affect renal perfusion and filtration capabilities. For instance, if an individual exhibits a high respiratory rate and low SpO₂, along with an elevated heart rate, it may suggest inadequate volume status or poor oxygenation. On the other hand, a combination of normal respiratory rate, SpO₂, and heart rate may indicate more favorable volume responsiveness. It should be noted that these findings provide valuable insights into the potential associations between selected parameters and VR in AKI patients. However, a comprehensive clinical evaluation and diagnostic investigations are required to determine the individualized causes and management strategies for AKI patients^27-29.

We compared the feature importance rankings obtained by random forest and gradient boosting algorithms. Both methods are based on tree-based models, which can capture the interaction effects among features. However, they differ in how they build and combine the trees. Gradient boosting is an additive model, which builds each tree on the residuals of the previous tree and assigns different weights to each tree^30-32. This means that gradient boosting can adjust the contribution of each feature or feature combination to the target variable more flexibly, and thus emphasize the importance of individual factors or factor combinations. Random forest is an averaging model, which builds each tree independently and averages the predictions of all trees^33-35. This means that random forest tends to balance the contribution of each feature or feature combination to the target variable, and thus emphasize the importance of different factors combined. These differences may have implications for interpreting the results and selecting the most relevant features for prediction.

limitation

One limitation of our study is the lack of data regarding the specific reasons for administering large-volume resuscitation. Since our study was not a pre-planned clinical trial, the indications for such fluid administration could not be predetermined. However, despite the absence of a prospective design, our study illuminates potential directions for future experimental designs through the analysis of large-scale data. Moreover, the parameters examined in this study, such as creatinine, blood urea nitrogen (BUN), blood glucose, bicarbonate, respiratory rate, SpO₂, Sbp, and Heart rate, are commonly measured clinical indicators with strong operational feasibility. By highlighting their association with oliguric AKI, our findings contribute valuable insights to guide clinical practice and stimulate further research in this field.

The consistent findings of both models, highlighting the significance of creatinine, blood urea nitrogen (BUN), blood glucose, bicarbonate, and age, suggest that a combination of these indicators can provide a clearer understanding of VU occurrence. By considering these features together, it becomes possible to identify potential warning signs more effectively. The variations in how the models build and combine trees have implications for interpreting the results and selecting the most relevant features for prediction.

In conclusion, this study utilized the MIMIC-IV2.2 database to develop predictive models for volume responsiveness in AKI patients. The Random Forest and GBM models outperformed the traditional logistic regression model in differentiating between volume-responsive and volume-unresponsive patients. Urinary creatinine, blood urea nitrogen, blood glucose, age, and bicarbonate were identified as important predictors of volume responsiveness. The findings suggest that machine learning techniques have the potential to improve the development and validation of predictive models in critical care research.

Ethics approval:

The original data collection obtained consent, and the institutional review board of the Massachusetts Institute of Technology (Cambridge, MA) approved the establishment of the database. Therefore, the ethical approval statement and informed consent were not required for this manuscript, however, due to the utilization of the MIMIC-IV database for our study, we did not seek additional ethical approval. However, we have completed the requirements of the COLLABORATIVE INSTITUTIONAL TRAINING INITIATIVE (CITI PROGRAM) and complied with its regulations. The coursework requirements of PART 1 OF 2 of the CITI Program have been fulfilled. We assure adherence to ethical standards in the handling and utilization of the data, with a commitment to maintaining its sensitivity and confidentiality. We will continue to adhere to the CITI Program and relevant legal regulations to ensure that our research is conducted within an ethical framework.

Code availability:

The databases were queried using Navicat Premium software (version 15.0), and computations were performed using R software (version 3.62). For more detailed information, researchers interested in the code can contact Dr. Yang(email: [email protected]).

Consent for publication

Not applicable.

Competing interests

All authors declare no competing interests.

Funding

This research received no funding.

Authors' contributions

Study design: H.Y.; data collection: H.Y.; data analysis: H.Y.; data interpretation: H.Y.; drafting the manuscript: H.Y., J.C.; revising the manuscript content: Y.Z., Y.W., and C.W.

Acknowledgements

We would like to extend our special gratitude to Dr. Zhongheng Zhang, from the Emergency Department of Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, for generously providing learning materials and insights in the field of machine learning. In previous studies, Zhongheng Zhang conducted extensive research on distinguishable indicators in VR AKI patients using the MIMIC-III database，and we have further demonstrated this conclusion based on the MIMIC-IV database. Dr. Zhang's pioneering research in the machine learning domain has greatly inspired our research group.

Availability of data and material

The data utilized in this study were obtained from the MIMIC-IV2.2 database (Medical Information Mart for Intensive Care IV), which is publicly available for authorized researchers. Access to the MIMIC-IV database can be requested through the PhysioNet website (https://physionet.org/). The fully anonymized data used in this study can also be downloaded from the following online address: https://figshare.com/s/ab182810ecd748251b3d

Ronco C, Bellomo R, Kellum JA. Acute kidney injury. Lancet 2019;394:1949-64.
Perez-Valdivieso JR, Bes-Rastrollo M, Monedero P, de Irala J, Lavilla FJ. Prognosis and serum creatinine levels in acute renal failure at the time of nephrology consultation: an observational cohort study. BMC Nephrol 2007;8:14.
Schneider J, Khemani R, Grushkin C, Bart R. Serum creatinine as stratified in the RIFLE score for acute kidney injury is associated with mortality and length of stay for children in the pediatric intensive care unit. Crit Care Med 2010;38:933-9.
Kellum JA, Lameire N, Aspelin P, et al. Kidney disease: improving global outcomes (KDIGO) acute kidney injury work group. KDIGO clinical practice guideline for acute kidney injury. #N/A 2012;2:1-138.
Marik PE, Lemson J. Fluid responsiveness: an evolution of our understanding. Br J Anaesth 2014;112:617-20.
Takala J. Volume responsive, but does the patient need volume? Intensive Care Med 2016;42:1461-3.
Kattan E, Castro R, Miralles-Aguiar F, Hernandez G, Rola P. The emerging concept of fluid tolerance: A position paper. J Crit Care 2022;71:154070.
Susantitaphong P, Cruz DN, Cerda J, et al. World incidence of AKI: a meta-analysis. Clin J Am Soc Nephrol 2013;8:1482-93.
Vanmassenhove J, Glorieux G, Hoste E, Dhondt A, Vanholder R, Van Biesen W. Urinary output and fractional excretion of sodium and urea as indicators of transient versus intrinsic acute kidney injury during early sepsis. #N/A 2013;17:R234.
Wlodzimirow KA, Abu-Hanna A, Royakkers AA, et al. Transient versus persistent acute kidney injury and the diagnostic performance of fractional excretion of urea in critically ill patients. Nephron Clin Pract 2014;126:8-13.
Dewitte A, Biais M, Petit L, et al. Fractional excretion of urea as a diagnostic index in acute kidney injury in intensive care patients. J Crit Care 2012;27:505-10.
Lievre L, Dupont V, Kanagaratnam L, et al. Urine-to-plasma creatinine ratio and fractional excretion of urea performance in acute-on-chronic kidney disease patients. J Nephrol 2023.
Cavallaro F, Sandroni C, Marano C, et al. Diagnostic accuracy of passive leg raising for prediction of fluid responsiveness in adults: systematic review and meta-analysis of clinical studies. Intensive Care Med 2010;36:1475-83.
Marik PE, Monnet X, Teboul JL. Hemodynamic parameters to guide fluid therapy. Ann Intensive Care 2011;1:1.
Muller L, Toumi M, Bousquet PJ, et al. An increase in aortic blood flow after an infusion of 100 ml colloid over 1 minute can predict fluid responsiveness: the mini-fluid challenge study. Anesthesiology 2011;115:541-7.
Marik PE, Desai H. Goal directed fluid therapy. Curr Pharm Des 2012;18:6215-24.
Johnson A, Bulgarelli L, Pollard T, Horng S, Celi L, Mark R. PhysioNet. 2021. MIMIC-IV[Google Scholar] 2023.
Zhang Z. Multiple imputation with multivariate imputation by chained equation (MICE) package. Ann Transl Med 2016;4:30.
Thomas ME, Blaine C, Dawnay A, et al. The definition of acute kidney injury and its use in practice. Kidney Int 2015;87:62-73.
Ostermann M. Diagnosis of acute kidney injury: Kidney Disease Improving Global Outcomes criteria and beyond. Curr Opin Crit Care 2014;20:581-7.
Bellomo R, Ronco C, Mehta RL, et al. Acute kidney injury in the ICU: from injury to recovery: reports from the 5th Paris International Conference. Ann Intensive Care 2017;7:49.
Waikar SS, Betensky RA, Emerson SC, Bonventre JV. Imperfect gold standards for kidney injury biomarker evaluation. J Am Soc Nephrol 2012;23:13-21.
Macedo E, Malhotra R, Claure-Del Granado R, Fedullo P, Mehta RL. Defining urine output criterion for acute kidney injury in critically ill patients. Nephrol Dial Transplant 2011;26:509-15.
Patel SS, Molnar MZ, Tayek JA, et al. Serum creatinine as a marker of muscle mass in chronic kidney disease: results of a cross-sectional study and review of literature. Journal of cachexia, sarcopenia and muscle 2013;4:19-29.
Higgins C. Urea and the clinical value of measuring blood urea concentration. Acutecaretesting Org 2016:1-6.
Böhmer AB, Just KS, Lefering R, et al. Factors influencing lengths of stay in the intensive care unit for surviving trauma patients: a retrospective analysis of 30,157 cases. #N/A 2014;18:1-10.
Prowle JR, Echeverri JE, Ligabo EV, Ronco C, Bellomo R. Fluid balance and acute kidney injury. #N/A 2010;6:107-15.
Cecconi M, Hofer C, Teboul J-L, et al. Fluid challenges in intensive care: the FENICE study: a global inception cohort study. Intensive Care Med 2015;41:1529-37.
Marik PE, Cavallazzi R. Does the central venous pressure predict fluid responsiveness? An updated meta-analysis and a plea for some common sense. Crit Care Med 2013;41:1774-81.
Friedman JH. Greedy function approximation: a gradient boosting machine. #N/A 2001:1189-232.
Chen T, Guestrin C. Xgboost: A scalable tree boosting system. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining; 2016. p. 785-94.
Elith J, Leathwick JR, Hastie T. A working guide to boosted regression trees. J Anim Ecol 2008;77:802-13.
Breiman L. Random forests. #N/A 2001;45:5-32.
Liu Y, Wang Y, Zhang J. New machine learning algorithm: Random forest. Information Computing and Applications: Third International Conference, ICICA 2012, Chengde, China, September 14-16, 2012 Proceedings 3; 2012: Springer. p. 246-52.
Louppe G. Understanding random forests: From theory to practice. arXiv preprint arXiv:14077502 2014.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Machine learning-based prediction model for volume responsiveness in critically ill patients with oliguric acute kidney injury

Status:

Version 1

Abstract

Figures

Background

Methods

Source of data

Participants

Outcome (volume responsiveness)

Predictors of VR-AKI

Model development

Statistical analysis

Results

Participants

The stepwise logistic regression model

Discussion

Conclusion

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1