Development and validation of machine learning models for glycemic variability in non-diabetic patients following cardiopulmonary bypass: a prospective observational study

doi:10.21203/rs.3.rs-4845952/v1

Download PDF

Article

Development and validation of machine learning models for glycemic variability in non-diabetic patients following cardiopulmonary bypass: a prospective observational study

https://doi.org/10.21203/rs.3.rs-4845952/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

There is a correlation between glucose variability (GV) after cardiopulmonary bypass (CPB) and major adverse events. Identifying early risk factors and developing a prediction model for preventing GV is crucial. No machine learning models have been developed for GV in non-diabetic patients during CPB cardiac operations. This study established six models: logistic regression (LR), random forest (RF), decision tree (DT), support vector machine (SVM), eXtreme gradient boosting (XGBoost), and categorical boosting (CatBoost). Each model was internally validated, and the SHAP method identified important variables. Among 360 non-diabetic patients, 213 (59.17%) developed GV in the ICU. The models showed AUC values from 0.7400 to 0.818 in the training set and from 0.6658 to 0.763 in the testing set. XGBoost performed best, with an AUC of 0.736, accuracy of 0.7798, sensitivity of 0.875, positive prediction value of 0.7778, F1-score of 0.8235, and Brier score of 0.2041. Postoperative insulin, BMI, intraoperative mean glucose, and CPB duration were crucial features. By combining XGBoost with SHAP, the developed models can be used to facilitate individualized risk evaluation, allowing timely intervention or targeted care.

Health sciences/Cardiology

Health sciences/Medical research

Glycemic variability

Cardiopulmonary Bypass

Intensive Care Units

Machine Learning

SHAP values

During open-heart surgery, cardiopulmonary bypass (CPB) is an essential life support technique ¹. The White Book of Chinese Cardiovascular Surgery and Extracorporeal Circulation in 2022² showed that 263,292 cardiovascular surgeries were performed. Of these, 159,949 were CPB surgeries, comprising 60.7% of all surgeries. A combination of factors including anaesthesia effects, surgical stress, extracorporeal circulation, and ischemia-reperfusion, led to the occurrence of systemic inflammatory response syndrome (SIRS) ³, which in turn promotes insulin resistance (IR) and causes acute hyperglycemia. Perioperative hyperglycemia can disrupt metabolic processes and organ dysfunction, exacerbating organ damage, triggering various complications, and increasing the risk of post-surgical infections and mortality ⁴. The prevalence of these complications in cardiac surgery is markedly higher compared to non-cardiac surgeries ⁵. Therefore, blood glucose management is crucial during the perioperative of cardiac surgery.

Glycemic variability (GV) is identified as a new indicator for evaluating glucose regulation, defined as the variability of glucose concentrations inside or outside the target range ^6,7. Research ⁸ suggests that these fluctuations might pose greater risks to health than chronic hyperglycemia. High GV reflects impaired glucose homeostasis, contributing to the development of cardiac fibrosis ⁹ and promoting the negative remodeling of the left ventricle ¹⁰. This can cause severe outcomes, including myocardial infarction, postoperative cardiac arrest, cerebrovascular accidents, renal failure, septic conditions, pneumonia, deep chest infections, repeated surgical interventions, and increased 30-day mortality rates ^11,12. Furthermore, patients exhibiting high GV are at an increased risk of experiencing hypoglycemia. This condition may lead to myocardial ischemia and arrhythmias as a result of sympathoadrenal system activation ^13,14. Data has shown that 60–80% of cardiac surgery patients, irrespective of their diabetes status, develop perioperative hyperglycemia, and about 60% of them are diagnosed with diabetes mellitus after a 1-year follow-up ¹⁵. The incidence of perioperative GV ranges from 20–50% in surgical patients and can reach up to 80% in those undergoing cardiac surgery ^5,16. Study ¹⁷ shown that non-diabetic patients with hyperglycemia experience more complications, extended hospitalizations, increased readmissions within 30 days, and higher hospital costs. This may be explained by the possibility that diabetic patients, having experienced glycemic fluctuations throughout their illness, may tolerate a wider range of blood sugar concentrations compared to non-diabete patients ^18,19. Furthermore, inadequate surveillance of blood glucose in non-diabetic patients may lead to the overlooked incidence of acute hyperglycemia and GV. Despite the critical role of the ICU in patients survival, it is also affected by issues such as constrained healthcare resources, staff shortages, and heavy economic burden ²⁰. Therefore, identifying risk factors and developing predictive models are crucial for preventing GV in non-diabetic patients.

In recent years, advancements in machine learning (ML) techniques have facilitated the development of predictive models for better glycemic management by estimating blood glucose levels. ML is an emerging field within medicine, that can be used to create algorithms capable of representation, adaptation, learning, prediction, and data analysis ^21,22. However, the majority of available ML-derived algorithms were derived based on outpatient diabetic patients ^23,24, mixed ICU patients²⁵ or patients with specific types of diabetes ²⁶. These models often rely on factors like physical activity, carbohydrate intake tracking, and self-blood sugar monitoring, which may not be readily available for most ICU patients recovering from cardiac surgery. In addition, the models themselves tend to prioritize avoiding hypoglycemia in their assessments ^27–30, or postoperative hyperglycemia (PHG) ³¹, or sustained dysglycemia (blood glucose < 72mg/dL or > 270 mg/dL on two consecutive days of admission) ³². In addition, although the ML techniques has shown potential to construct models, it fails to offer direct interpretation, a challenge often characterized as the “black-box” issue ³³. Currently, no study has explored ML models for GV in non-diabetic patients undergoing cardiac surgery with CPB. Because there is no single best way to choose a predictive model for specific clinical situations, we need to compare different models to find the most effective one ³⁴.

This study aim to develop and validate six ML models intended to predict GV in non-diabetic patients undergoing cardiac surgery with CPB in the initial 24-hour following ICU admission. Furthermore, the SHAP approach was utilized to assess the importance of various variables and provide explanations for the model.

2.1 Study setting

A prospective, observational study was conducted at a tertiary general hospital in Fuzhou, China. Patients who underwent CPB necessitating ICU admission between November 2022 and 31 December 2023 were enrolled in the study. The Ethics committee of Fujian Provincial Hospital reviewed and approved the study (No. K2020-09-035). In compliance with the Declaration of Helsinki, every participant signed a written informed consent form prior to enrollment in the study. Case data for each participant were collected anonymously with approval from hospital management and adhering to the institutional guidelines and procedures.

2.2 Study population

The inclusion criteria were: (1) cardiac surgery with CPB; (2) age ≥ 18 years; (3) an admission to the ICU and stay at least 24 hours; (4) without previous history of diabetes. The exclusion criteria: (1) fasting plasma glucose (FPG) ≥ 7.0mmol/L or glycated hemoglobin A_1c (HbA_1c) ≥ 6.5%; (2) patients undergoing urgent operations, multiple surgeries within 24 hours, or surgical cancellations; (3) patients received glucocorticoid treatment before surgery; (4) presence of other disorders affecting glucose metabolism, such as Cushing’s syndrome, acute metabolic disorders, hyperthyroidism, acromegaly, or other stress-related conditions. Diabetes was defined by one or more of the following criteria: fasting glucose levels exceeding 7.0 mmol/L, non-fasting glucose levels above 11.1 mmol/L, HbA_1c levels over 6.5%, the use of oral antidiabetic agents, or self-reported diagnosis ³⁵. Those who did not fit these descriptions were defined as non-diabetic. Currently, consensus is lacking regarding the minimum sample size necessary for machine learning models ³⁶. The prevalence of GV was 34% based on a pre-investigation, with an allowable error (δ) of 0.15p, the minimum sample size was calculated to be 332 for this study.

2.3 Outcome measure

The primary outcome measure was GV occurring within the first 24-hour period following admission to the ICU. GV assessment utilized the Standard Deviation of Blood Glucose (SDBG), which is defined as the standard deviation of either capillary or venous blood glucose measurements, indicative of daily fluctuations in blood glucose levels ³⁶. SDBG was calculated as the square root of the mean of the squared differences between individual glucose measurements and the mean glucose level ³⁷. The SDBG was calculated from the measured glucose values, with a normal reference value of < 2.0 mmol/L.

2.4 Predictor variables

To identify the risk factors associated with GV, a comprehensive literature search was conducted across multiple academic platforms, such as PubMed, Web of Science, Embase, the Cochrane Library, as well as Chinese databases CNKI, VIP, and Wanfang Med Online. After several rounds of discussion by a team of researchers, the predictive variables of GV were selected by considering published articles, clinical insights and pathophysiological considerations. The clinical variables included were: (1) preoperative variables: gender, age, body mass index (BMI), history of smoking, history of drinking, hypertension, cerebrovascular disease, cardiovascular disease, New York Heart Association (NYHA) classification, left ventricular ejection fraction (LVEF), preoperative alanine aminotransferase (ALT), preoperative aspartate aminotransferase (AST), preoperative glucose. (2) intraoperative variables: type of surgery, American Society of Anesthesiologists (ASA) classification, duration of surgery, duration of anesthetizing, duration of CPB, duration of aortic block, allogeneic blood transfusion, intraoperative blood loss, intraoperative insulin, epinephrine, norepinephrine, dopamine, dobutamine, milrinone, isoproterenol, intraoperative mean glucose. (3) postoperative variables: Acute Physiologic Assessment and Chronic Health Evaluation II (APACHE-II), Sequential Organ Failure Assessment (SOFA), postoperative corticosteroids, postoperative insulin, postoperative lactic acid, postoperative White Cell Rate (WBC). Upon enrollment in the study, the survey team collected clinical data from the hospital’s electronic medical record system using their self-designed clinical record forms (CRFs).

2.5 Definition of the predictors

All measurements were conducted by trained researchers who were unaware of the study’s objectives to avoid bias.

Gender, age, body mass index (BMI), history of smoking, history of drinking, hypertension, cerebrovascular disease, cardiovascular disease, New York Heart Association (NYHA) classification and left ventricular ejection fraction (LVEF) were collected upon admission. (2) Alanine aminotransferase (ALT), aspartate aminotransferase (AST), preoperative glucose were measured the day before surgery. (3) Type of surgery, American Society of Anesthesiologists (ASA) classification, duration of surgery, duration of anesthetizing, duration of CPB, duration of aortic block, allogeneic blood transfusion, intraoperative blood loss, intraoperative insulin, epinephrine, norepinephrine, dopamine, dobutamine, milrinone, isoproterenol and intraoperative mean glucose were collected through the electronic anesthesia medical record after the surgery. (4) Postoperative lactic acid was measured immediately through blood gas analysis after admission to the ICU. (5) Postoperative White Cell Rate (WBC) were measured immediately after admission to the ICU. (6) Postoperative corticosteroids and postoperative insulin were collected after 24h admission to the ICU. (7) Acute Physiologic Assessment and Chronic Health Evaluation II (APACHE-II), Sequential Organ Failure Assessment (SOFA) were assess by ICU doctor.

2.6 Glucose management

(1) Preoperative blood glucose: Blood glucose assessments were conducted immediately after patient enrolment in the study. Fasting serum glucose was measured the day before surgery. (2) Intraoperative blood glucose: Anesthesiologists routinely collected the first arterial blood gas sample 15 minutes following intubation. Blood-gas analysis was conducted using an ABL90 FLEX blood-gas Analyser™ (Radiometer, Denmark). Subsequently, samples were collected multiple times based on surgery length and vital signs of the patient. The average value of all intraoperative arterial blood glucose levels was calculated and recorded to two decimal places. (3) Postoperative blood glucose: Blood glucose measurements were conducted bi-hourly using a glucometer (Baiankang 1816, Leverkusen, Germany) using fingertip capillary blood collected by a nurse during the ICU stay. Should glucose levels deviate from the target range, the frequency of monitoring was increased ³⁸. To ensure consistency, all participating nurses received training from the same instructor using identical materials. The training covered essential skills like using glucometers, collecting blood samples, interpreting and recording results, managing blood sugar levels, and handling emergency situations. Blood glucose management was determined by the attending physician according to the glucose control protocol of the ICU. Typically, the target range for perioperative blood glucose was between 6.0 mmol/L and 10.0 mmol/L ³⁹, with an increase of up to 12.0 mmol to prevent hypoglycemia during surgery. Insulin was administered via continuous intravenous infusion to manage blood glucose levels. To minimize potential bias, individuals responsible for glucose monitoring and management were unaware that the data they collected would be used for this research study.

2.7 Missing data handling

To avoid bias caused by exclusion of patients with missing data, we employed multiple imputation techniques for variables with missing values under 30%. It was assumed that the missing data were randomly distributed and imputed using a fully conditional specification method via the “mice” package (version 3.13.0) in R (version 4.2.1).

2.8 Machine learning development process

Patients were randomly divided into a training set and a testing set at a 7:3 ratio, including 252 in the training set and 108 in the testing set. Six ML models including logistic regression (LR), random forest (RF), decision tree (DT), support vector machine (SVM), eXtreme Gradient boosting (XGBoost) and categorical boosting (CatBoost) were used to construct predictive models. We employed five-fold internal cross-validation to optimize the training set and determine the best hyperparameters for each model. The performance of the models across both datasets was evaluated using various metrics: area under the receiver operator characteristic curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1-score. Calibration of the models was assessed using the Brier score and calibration curves, while the decision curve analysis (DCA) was applied to assess the clinical utility.

2.9 Machine learning explainable tool

As an explainable AI technique, SHAP is utilized to explain the outcomes of ML models ⁴⁰. The SHAP approach was utilized to assess the influence of each variable on the model. The SHAP values illustrate the positive or negative contributions of each predictor to the target variable. Furthermore, the interpretation of each data point can be determined based on its unique SHAP values. The flowchart depicting the process of model derivation is presented in Fig. 1.

2.10 Statistical analysis

Statistical analyses and calculations were performed using R software (version 4.2.1). Each variable in both the training and testing sets were compared. Continuous variables were presented as medians (IQRs) and compared using the Mann-Whitney U-test. Nominal variables were presented as counts (percentages) and compared using the chi-squared. Statistical significance was defined as p < 0.05 on both sides.

3.1 Study population

A total of 865 CPB surgeries were performed in the study. Of these surgeries, 360 patients met the study’s inclusion criteria. Of these patients, 213 (59.17%) exhibited occurrences of GV. The differences in baseline characteristics among the training and testing sets were not statistically significant (p > 0.05), as shown in Table 1. Table S1 displays the characteristics of patients who occured GV compared to those who did not. In univariate analysis, variables such as gender, preoperative ALT, intraoperative mean glucose, intraoperative insulin, intraoperative epinephrine, intraoperative allogeneic blood transfusion, postoperative insulin, APACHE-II, SOFA and postoperative lactic acid were considered to be significantly different between the two groups.

Table 1

Comparison of the clinical characteristics of all patients in the training and testing sets
Clinical characteristics	Total (N = 360)	Testing (n = 252)	Training (n = 108)	P value
Gender, n(%)				0.262
Female	159 (44.2)	106 (42.2)	53 (48.6)
Male	201 (55.8)	145 (57.8)	56 (51.4)
Age (years), median (IQR)	56.00 (50.00, 64.00)	57.00 (50.00, 64.00)	55.00 (50.00, 63.00)	1
BMI, median (IQR)	22.98 (20.39, 25.09)	23.12 (20.61, 24.88)	22.40 (20.28, 25.71)	0.529
History of smoking, n(%)				0.923
Smoking	49 (13.6)	35 (13.9)	14 (12.8)
Never smoked	290 (80.6)	202 (80.5)	88 (80.7)
Smoking cessation for ≥ 1 year	21 (5.8)	14 (5.6)	7 (6.4)
History of drinking, n(%)				0.745
Drinking	28 (7.8)	18 (7.2)	10 (9.2)
Never drinked	326 (90.6)	229 (91.2)	97 (89.0)
Drinking cessation for ≥ 1 year	6 (1.7)	4 (1.6)	2 (1.8)
Hypertension, n(%)	112 (31.1)	76 (30.3)	36 (33.0)	0.605
Cerebrovascular disease, n(%)	25 (6.9)	16 (6.4)	9 (8.3)	0.519
Cardiovascular disease, n(%)	77 (21.4)	53 (21.1)	24 (22.0)	0.848
NYHA, n(%)				0.556
Class I	2 (0.6)	1 (0.4)	1 (0.9)
Class Ⅱ	47 (13.1)	35 (13.9)	12 (11.0)
Class III	250 (69.4)	170 (67.7)	80 (73.4)
Class Ⅳ	61 (16.9)	45 (17.9)	16 (14.7)
LVEF, median (IQR)	58.00 (54.00, 62.00)	58.00 (55.00, 61.00)	58.00 (54.00, 62.00)	0.468
Preoperative ALT, median (IQR)	21.00 (14.00, 33.25)	20.00 (14.00, 36.00)	21.00 (14.00, 31.00)	0.903
Preoperative AST, median (IQR)	23.00 (18.00, 32.25)	23.00 (18.00, 34.50)	23.00 (18.00, 31.00)	0.549
Preoperative glucose, median (IQR)	5.09 (0.74)	5.07 (0.74)	5.14 (0.75)	0.392
Type of surgery, n(%)				0.992
Cardiac valve surgery	283 (78.6)	198 (78.9)	85 (78.0)
CABG	18 (5.0)	12 (4.8)	6 (5.5)
Major vascular surgery	30 (8.3)	21 (8.4)	9 (8.3)
Other cardiac surgery	29 (8.1)	20 (8.0)	9 (8.3)
ASA, n(%)				0.459
Class III	4 (1.1)	4 (1.6)	0 (0.0)
Class Ⅳ	351 (97.5)	244 (97.2)	107 (98.2)
Class V	5 (1.4)	3 (1.2)	2 (1.8)
Duration of surgery, median (IQR)	330.00 (285.00, 405.00)	325.00 (287.50, 395.00)	340.00 (280.00, 429.00)	0.506
Duration of anesthesiaing, median (IQR)	410.00 (360.00, 476.25)	410.00 (360.00, 470.00)	410.00 (360.00, 500.00)	0.512
Duration of CPB, median (IQR)	189.00 (148.50, 235.00)	188.00 (148.50, 228.50)	191.00 (149.00, 238.00)	0.63
Duration of aortic block, median (IQR)	113.00 (87.75, 145.00)	113.00 (88.00, 145.00)	112.00 (86.00, 145.00)	0.987
Allogeneic blood transfusiony, n(%)	147 (40.8)	99 (39.4)	48 (44.0)	0.415
Intraoperative mean glucose, median (IQR)	8.80 (1.62)	8.81 (1.67)	8.78 (1.50)	0.868
Intraoperative blood loss, ml, median (IQR)	700.00 (500.00, 1000.00)	650.00 (500.00, 1000.00)	800.00 (500.00, 1000.00)	0.663
Intraoperative insulin, n(%)	29 (8.1)	18 (7.2)	11 (10.1)	0.35
Intraoperative epinephrine, n(%)	338 (93.9)	233 (92.8)	105 (96.3)	0.203
Intraoperative norepinephrinee, n(%)	217 (60.3)	150 (59.8)	67 (61.5)	0.761
Intraoperative dopamine, n(%)	47 (13.1)	35 (13.9)	12 (11.0)	0.448
Intraoperative dobutamine, n(%)	39 (10.8)	30 (12.0)	9 (8.3)	0.3
Intraoperative milrinone, n(%)	158 (43.9)	113 (45.0)	45 (41.3)	0.512
Intraoperative isoproterenol, n(%)	16 (4.4)	11 (4.4)	5 (4.6)	1
APACHE II, median (IQR)	26.00 (22.00, 31.00)	26.00 (22.00, 31.00)	26.00 (23.00, 31.00)	0.391
SOFA, median (IQR)	12.00 (10.00, 14.00)	12.00 (10.00, 13.50)	12.00 (10.00, 14.00)	0.976
Postoperative corticosteroids, n(%)	326 (90.6)	226 (90.0)	100 (91.7)	0.612
Postoperative insulin, n(%)	272 (75.6)	192 (76.5)	80 (73.4)	0.53
Postoperative WBC, 10^9/L, median (IQR)	17.80 (13.80, 21.90)	17.60 (13.90, 21.90)	18.20 (13.70, 21.60)	0.966
Postoperative lactic acid, median (IQR)	6.00 (3.90, 8.90)	6.00 (4.05, 8.70)	6.00 (3.80, 9.10)	0.946
GV	213 (59.2)	149 (59.4)	64 (58.7)	0.909

3.2 Model performance comparisons

Table 2 displays the performance of six ML models. In the training set, the AUC values spanned from 0.7400 to 0.818, while in the testing set, these values varied from 0.6658 to 0.763. Figure 2 illustrates the ROC curves for the models. The random forest and decision tree models showed the best performance in the training set, whereas the XGBoost and SVM models showed best performance in the testing set. Although the AUC values indicate the models’ predictive accuracy, they do not indicate the clinical applicability or which one of the two is more preferable ^41,42. To further clarify this, calibration curves and DCA were analyzed. The Brier score of the calibration curve ranged from 0.1594 to 0.2170 in the training set and from 0.1928 to 0.2269 in the testing set. The Brier scores were all below 0.25, indicating that the constructed models were reliable. The DCA suggested that these models had a good clinical utility (Fig. 3). Comprehensive analysis demonstrated that the XGBoost model was the optimal model, achieving an AUC of 0.763 (95%CI 0.6648–0.8613) in the testing set, with 0.7798 accuracy, 0.875 sensitivity, 0.6444 specificity, 0.7778 PPV and 0.7838 NPV. Calibration curves and DCA for the other models are presented in Supplementary Figs. S1 to S4. Compared to these models, the XGBoost model achieved the highest F1 score of 0.8235. This F1 score, which balances precision and recall, indicates that the XGBoost model has good predictive performance. Table S2 contains the optimal hyperparameters for each model.

Table 2

Model performance in the training and testing set.
	AUC (95%CI)	Accuracy	Sensitivity	Specificity	PPV	NPV	F1	Brier
Training set
LR	0.7400(0.6769–0.8031)	0.7171	0.5686	0.8188	0.7349	0.6824	0.7746	0.1978
RF	0.8108(0.7576–0.8639)	0.7371	0.7843	0.7047	0.8268	0.6452	0.7609	0.2012
DT	0.8003(0.7443–0.8563)	0.7928	0.7059	0.8523	0.8089	0.7660	0.8301	0.1594
SVM	0.7706(0.7106–0.8307)	0.7131	0.8627	0.6107	0.8667	0.6027	0.7165	0.2056
XGBoost	0.7915(0.7341–0.8489)	0.7490	0.7255	0.7651	0.8028	0.6789	0.7835	0.2027
CatBoost	0.7866(0.7289–0.8444)	0.7291	0.7059	0.7450	0.7872	0.6545	0.7655	0.2170
Testing set
LR	0.7493(0.6518–0.8468)	0.7615	0.5333	0.9219	0.7375	0.8276	0.8194	0.1928
RF	0.7205(0.6203–0.8207)	0.7156	0.6444	0.7656	0.7538	0.6591	0.7597	0.2189
DT	0.6658(0.5562–0.7754)	0.7156	0.5333	0.8438	0.7200	0.7059	0.7770	0.2269
SVM	0.7573(0.662–0.8526)	0.7615	0.6667	0.8281	0.7794	0.7317	0.8030	0.2015
XGBoost	0.763(0.6648–0.8613)	0.7798	0.6444	0.8750	0.7778	0.7838	0.8235	0.2041
CatBoost	0.7472(0.6493–0.8452)	0.7523	0.6000	0.8594	0.7534	0.7500	0.8029	0.2189

It presents the inclusion criteria for patients, division of data into training and testing sets, parameter training of the six models, performance validation and explanation.

X-axis indicates the threshold probability and Y-axis indicates the net benefit. The black line represents the assumption that all patients undergo intervention, whereas the red line shows that none of the patients undergo intervention.

A represent the weights of variables importance in the XGBoost model. B represent the SHAP is utilized to interpret the XGBoost model's outputs. The characteristics are depicted in a SHAP summary plot where each horizontal line corresponds to a distinct feature, and the x-axis quantifies the SHAP values associated with each feature. Yellow dots represent higher eigenvalues and purple dots represent lower eigenvalues.

3.3 Explanation of the XGBoost Model via SHAP Approach

The SHAP approach was employed to determine the importance of predictor variables for GV in the XGBoost model. Figure 4A shown the importance of predictor variables, ranked by the contribution to the model’s predictions. The most critical determinant was the postoperative insulin, which had the strongest predictive value for all prediction horizons. This was closely followed by variables including the intraoperative mean glucose, BMI, duration of CPB, and APACHE-II. In the study, we employed SHAP values to elucidate the relationship between the predictor variables and GV. As presented in Fig. 4B, the sign of the SHAP value determines the direction of its contribution, whereas the magnitude of the SHAP value reflects its importance ⁴³. The SHAP value is represented by a color gradient. Color from purple to yellow indicate the SHAP values range from low to high. We can see that postoperative insulin use, decreased BMI, higher intraoperative mean glucose, and longer duration of CPB would elevate the formation of GV in non-diabetic patients undergoing cardiac surgery with CPB.

In this study, we development six ML models for predicting the risk of GV occurrence within the first 24 hours postoperatively following CPB. Among these models, the XGBoost model demonstrated the best predictive performance. Consequently, the XGBoost model was chosen as the optimal model for the early identification of GV risk in patients post-CPB surgery. The XGBoost model showed acceptable accuracy in predicting glycemic variability risk. This allows for proactive interventions to improve patient outcomes.

Over the past decade, the application of machine learning techniques to redict anomalous blood glucose levels has grown extensively. Previously, various methodologies were employed, including neural network ^44,45, linear model with multiple input variables ²⁴, and mathematical model utilizing both the first and second derivatives of the continuous glucose monitoring (CGM) data ⁴⁶ or incorporating constant endogenous glucose production along with other physiological parameters for real-time application to develop prediction models ⁴⁷. The choice of how we define the research question and select the study population significantly impacts a model's generalizability and usefulness in clinical settings. At present, there is a lack of predictive models for GV specifically in non-diabetic patients undergoing CPB.

In this study, the SHAP values were used to interpret the outputs of the XGBoost model and identified several important variables associated GV in non-diabetic patients undergoing cardiac surgery with CPB. This approach helps in understanding how each feature in the dataset influences the model’s predictions, providing a clearer insight into the decision-making process of the machine learning model. Postoperative insulin use has been recognized as the most important variable in the model. The results indicated that postoperative insulin use increased the risk of GV. The univariate analysis (Tabe S1) also supports the finding that patients who used insulin postoperative had higher SDBG. For postoperative patients with blood sugar exceeding 10.0 mmol/L, intravenous insulin infusion is the preferred treatment. Unlike injections, intravenous delivery sends insulin directly into the bloodstream, allowing it to reach tissues quickly and lower blood sugar levels. However, overuse of insulin can lead to hypoglycemia ⁴⁸. Strong glucose control methods can lead to hypoglycemia and wider glucose fluctuations. The study by Sanjay OP et al. ⁴⁹ found that attempting to maintain normoglycemia with insulin during CPB might lead to postoperative hypoglycemia. Currently, expert opinions from various countries differ slightly regarding the requirements for blood glucose control targets during the perioperative period. Based on the available literature ^50,51, it is generally accepted that perioperative target blood glucose should be controlled to a range of 7.8 to 10 mmol/L. The international guidelines also supplement that the minimum target range for blood glucose can be set from 4.4 to 8 mmol/L ^51,52. However, the optimal approach to glucose regulation remains unclear. Future studies with more patients and comprehensive glucose monitoring data are necessary to solidify the link between glycemic variability and insulin administration.

BMI was also identified as an important variable. This study was consistent with the findings by Wang et al. ⁵³, which demonstrated that lower BMI correlated with increased GV. This may be ascribed to the possibility that individuals with lower BMI have poorer beta-cell function compared to those who are overweight or obese. Moreover, high BMI was associated with insulin resistance. In contrast, lower BMI was primarily associated with insulin deficiency, which makes controlling blood glucose with medication or insulin more difficult. Therefore, we propose monitoring of blood glucose in patients with lower BMI to prevent GV.

Meanwhile, we observed that higher intraoperative mean glucose levels were associated with an increased risk of GV, confirming the results obtained by Cornelia Knaak et al ⁵⁴. In the study, they found a significant rise in both mean (p < 0.001) and maximum BG levels (p = 0.001) postoperatively in patients experiencing intraoperative dysglycemia. It is widely recognized that surgical procedures induce a stress response, which in turn prompts the secretion of both catecholamines and cortisol. These hormones can cause temporary insulin resistance, leading to a condition called stress hyperglycemia ⁵⁴. Despite this, the incidence of intraoperative hyperglycemia is often underestimated in non-diabetic patients. Therefore, ideal clinical blood glucose management should focus not only on postoperative blood glucose levels and complications, but also on the trends of intraoperative blood glucose change to reduce the postoperative GV.

Our study also identified the duration of CPB as a significant predictor of GV. Patients undergoing longer CPB durations experienced a greater increase in GV. This association likely relates to the stress response that invariably occurs during cardiac surgery with CPB. The longer the duration of CPB, the stronger the stress response in the body, which increases systemic inflammatory response and insulin resistance. This decreases sensitivity of body tissues to insulin, triggering increased insulin usage. In addition, patients undergoing CPB are routinely exposed to hypothermia. Cueni-Villoz N et al. ⁵⁵ reported that hypothermia increased the levels of blood glucose concentrations, elevated GV, and enhanced insulin requirement, which was consistent with our findings. With the improvements in surgical techniques, the operative time, CPB time and cross-clamp time are expected to be significantly reduced. This may effectively improve the body’s internal environment and decrease perioperative GV.

This study has several advantages. Firstly, our study focused on non-diabetic patients. According to Krinsley JS ⁵⁶ and Hao-ming Hestudy et al. ⁵⁷, GV was significantly associated with mortality and had the poorest prognosisin in the non-diabetic population. Another study ⁵⁸ also supports this finding, indicating that an increase in GV is associated with a higher risk of mortality in non-diabetic patients, but this is not necessarily the case for diabetic patients. Thus, it appears that non-diabetes patients exhibit less tolerance to high glucose variation compared to those with diabetes. Currently, there are no reliable predictive models for GV in non-diabetes patients. Thus, this study can improve the management of postoperation glycemia in non-diabetes patients. Secondly, we established the XGBoost technique, with can rapid computation, good generalization and excellent predictive capabilities ^59–61. XGBoost, unlike traditional machine learning models, employs an ensemble of decision trees. The outputs of all decision trees are combined to create the final output of the XGBoost model ⁶². XGBoost has enabled the development of new generation medical applications, from accurate diagnosis to personalized patient management. These applications hold immense potential to improve real-world healthcare outcomes ^63–65. Here, we found that XGBoost could predict GV in non-diabetic patients undergoing cardiac surgery with CPB. The third advantage of this study is that we employed SHAP values to visually explain the selected variables. Despite the high accuracy of ML algorithms, it is limited by difficult interpretability, known as "black box"⁶⁶. The SHAP algorithm showed good ability to address this problem. It ranked each feature’s importance within the model and illustrated how different variables contribute to predictive outcomes.

This study has some limitations. Firstly, the data were collected from a single institution with a relatively small sample size. To reduce overfitting and maximize the models’ advantages, larger sample sizes should be enrolled in future studies. Secondly, to improve the generalization of our models, further studies are needed to test the generalization of the models in independent external validation dataset from multiple institutions. Thirdly, no gold standard inclusion or exclusion criteria have been proposed for GV. Fourthly, we monitored postoperative glucose levels only during the initial 24 hours, neglecting potential impacts from subsequent fluctuations. Moreover, we excluded 58.38% of the initial participants, which might affect the generalizability of our findings.

Using several critical perioperative variables, we constructed an interpretable XGBoost model for improving the clinical diagnosis and prediction. This model showed good potential to identify non-diabetic patients who are at high risk of GV after CPB. This allows for early intervention and establishment of better management strategies. However, calculating GV using traditional methods can be challenging in the perioperative period. The emergence of CGM presents a significant opportunity. Future research will likely focus on developing predictive models that utilize CGM data to improve patient care.

BMI, Body Mass Index; NYHA, New York Heart Association; LVEF, Left Ventricular Ejection Fraction; AST, Aspartate aminotransferase; ALT, Alanine aminotransferase; CABG, Coronary Artery Bypass Graft; ASA, American Society of Anesthesiologists; CPB, Cardiopulmonary Bypass; APACHE II, Acute Physiologic Assessment and Chronic Health Evaluation II; SOFA, Sequential Organ Failure Assessment; WBC, White Cell Rate; LR, logistic regression; RF, random forest; DT, decision tree; SVM, support vector machine; XGBoost, eXtreme gradient boosting; CatBoost, categorical boosting; PPV, positive predictive value; NPV, negative predictive value; SHAP, Shapley Additive Explanations.

Author contributions

Shanshan Wang: Conceptualization, Methodology, Data Curation, Formal analysis, Validation, Writing - Original Draft. Ruiyan Zhuo: Methodology, Investigation, Data Curation, Writing - Original Draft. Xiuxia Lin: Investigation. Nan Wang: Investigation, Data Curation. Yuqing Xu: Investigation. Na Li: Conceptualization, Funding acquisition, Supervision,Writing – review & editing.

Data availability

The data that support the fndings of this study are available on request from the corresponding author upon reasonable request.

Funding

This work was supported by the Fujian Provincial Health Commission Young and Middle-aged Talents Training Project (No. 2020GGB008). The sponsors had no role in the study design and collection, analysis, and interpretation of data; writing of the report; or decision to submit the article for publication.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Conflict of interest

The authors have no conﬂicts of interest to disclose.

Pan, T. et al. The low-dose colchicine in patients after non-CABG cardiac surgery: a randomized controlled trial. Crit Care 27, 49 (2023).
Xing, H. White book of Chinese cardiovascular surgery and extracorporeal circulation in 2020. Chinese Journal of Extracorporeal Circulation 21, 197–200 (2023).(In Chinese)
Adamik, B., Kübler, A., Gozdzik, A. & Gozdzik, W. Prolonged Cardiopulmonary Bypass is a Risk Factor for Intestinal Ischaemic Damage and Endotoxaemia. Heart Lung Circ 26, 717–723 (2017).
Palermo, N. E. & Garg, R. Perioperative Management of Diabetes Mellitus: Novel Approaches. Curr Diab Rep 19, 14 (2019).
Duggan, E. W., Carlson, K. & Umpierrez, G. E. Perioperative Hyperglycemia Management: An Update. Anesthesiology 126, 547–560 (2017).
Service, F. J. Glucose variability. Diabetes 62, 1398–1404 (2013).
Suh, S. & Kim, J. H. Glycemic Variability: How Do We Measure It and Why Is It Important? Diabetes Metab J 39, 273–282 (2015).
Ceriello, A. et al. Oscillating glucose is more deleterious to endothelial function and oxidative stress than mean glucose in normal and type 2 diabetic patients. Diabetes 57, 1349–1354 (2008).
Saito, S. et al. Glucose fluctuations increase the incidence of atrial fibrillation in diabetic rats. Cardiovasc Res 104, 5–14 (2014).
Gohbara, M. et al. Glycemic Variability Determined by Continuous Glucose Monitoring System Predicts Left Ventricular Remodeling in Patients With a First ST-Segment Elevation Myocardial Infarction. Circ J 79, 1092–1099 (2015).
Subramaniam, B. et al. Increased glycemic variability in patients with elevated preoperative HbA1C predicts adverse outcomes following coronary artery bypass grafting surgery. Anesth Analg 118, 277–287 (2014).
Clement, K. C. et al. Increased Glucose Variability Is Associated With Major Adverse Events After Coronary Artery Bypass. Ann Thorac Surg 108, 1307–1313 (2019).
Desouza, C., Salazar, H., Cheong, B., Murgo, J. & Fonseca, V. Association of hypoglycemia and cardiac ischemia: a study based on continuous monitoring. Diabetes Care 26, 1485–1489 (2003).
Reno, C. M. et al. Severe hypoglycemia-induced lethal cardiac arrhythmias are mediated by sympathoadrenal activation. Diabetes 62, 3570–3581 (2013).
Galindo, R. J., Fayfman, M. & Umpierrez, G. E. Perioperative Management of Hyperglycemia and Diabetes in Cardiac Surgery Patients. Endocrinol Metab Clin North Am 47, 203–222 (2018).
Shah, N. J., Leis, A., Kheterpal, S., Englesbe, M. J. & Kumar, S. S. Association of intraoperative hyperglycemia and postoperative outcomes in patients undergoing non-cardiac surgery: a multicenter retrospective study. BMC Anesthesiol 20, 106 (2020).
Buehler, L. et al. The impact of hyperglycemia and obesity on hospitalization costs and clinical outcome in general surgery patients. J Diabetes Complications 29, 1177–1182 (2015).
Krinsley, J. S. et al. The Interaction of Acute and Chronic Glycemia on the Relationship of Hyperglycemia, Hypoglycemia, and Glucose Variability to Mortality in the Critically Ill. Crit Care Med 48, 1744–1751 (2020).
Sechterberger, M. K. et al. The effect of diabetes mellitus on the association between measures of glycaemic control and ICU mortality: a retrospective cohort study. Crit Care 17, R52 (2013).
Kramer, A. A., Dasta, J. F. & Kane-Gill, S. L. The Impact of Mortality on Total Costs Within the ICU. Crit Care Med 45, 1457–1463 (2017).
Handelman, G. S. et al. eDoctor: machine learning and the future of medicine. J Intern Med 284, 603–619 (2018).
Greener, J. G., Kandathil, S. M., Moffat, L. & Jones, D. T. A guide to machine learning for biologists. Nat Rev Mol Cell Biol 23, 40–55 (2022).
Oviedo, S., Vehí, J., Calm, R. & Armengol, J. A review of personalized blood glucose prediction strategies for T1DM patients. Int J Numer Method Biomed Eng 33, 10 (2017).
Zecchin, C., Facchinetti, A., Sparacino, G. & Cobelli, C. How Much Is Short-Term Glucose Prediction in Type 1 Diabetes Improved by Adding Insulin Delivery and Meal Content Information to CGM Data? A Proof-of-Concept Study. J Diabetes Sci Technol 10, 1149–1160 (2016).
Pappada, S. M. et al. Evaluation of a model for glycemic prediction in critically ill surgical patients. PLoS One 8, e69475 (2013).
Mayo, M., Chepulis, L. & Paul, R. G. Glycemic-aware metrics and oversampling techniques for predicting blood glucose levels using machine learning. PLoS One 14, e0225613 (2019).
Ena, J. et al. Derivation and validation model for hospital hypoglycemia. Eur J Intern Med 47, 43–48 (2018).
Shah, B. R., Walji, S., Kiss, A., James, J. E. & Lowe, J. M. Derivation and Validation of a Risk-Prediction Tool for Hypoglycemia in Hospitalized Adults With Diabetes: The Hypoglycemia During Hospitalization (HyDHo) Score. Can J Diabetes 43, 278-282.e1 (2019).
Horton, W. B., Barros, A. J., Andris, R. T., Clark, M. T. & Moorman, J. R. Pathophysiologic Signature of Impending ICU Hypoglycemia in Bedside Monitoring and Electronic Health Record Data: Model Development and External Validation. Crit Care Med 50, E221–E230 (2022).
Ruan, Y. et al. Predicting the Risk of Inpatient Hypoglycemia With Machine Learning Using Electronic Health Records. Diabetes Care 43, 1504–1511 (2020).
Chen, Y. et al. The prognosis of patients with postoperative hyperglycemia after Stanford type A aortic dissection surgery and construction of prediction model for postoperative hyperglycemia. Front Endocrinol (Lausanne) 14, 1063496 (2023).
Kyi, M. et al. Clinical Prediction Tool To Identify Adults With Type 2 Diabetes at Risk for Persistent Adverse Glycemia in Hospital. Can J Diabetes 45, 114-121.e3 (2021).
Azodi, C. B., Tang, J. & Shiu, S. H. Opening the Black Box: Interpretable Machine Learning for Geneticists. Trends Genet 36, 442–455 (2020).
Clift, A. K. et al. Development and internal-external validation of statistical and machine learning models for breast cancer prognostication: cohort study. BMJ 381, e073800 (2023).
Tangri, N. et al. Multinational Assessment of Accuracy of Equations for Predicting Risk of Kidney Failure: A Meta-analysis. JAMA 315, 164–174 (2016).
Van Der Ploeg, T., Austin, P. C. & Steyerberg, E. W. Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Med Res Methodol 14, 137 (2014).
Breyton, A. E., Lambert-Porcheron, S., Laville, M., Vinoy, S. & Nazare, J. A. CGMS and Glycemic Variability, Relevance in Clinical Research to Evaluate Interventions in T2D, a Literature Review. Front Endocrinol (Lausanne) 12, (2021).
Jinjing, W. et al. Chinese clinical practice guidelines for perioperative blood glucose management. Diabetes Metab Res Rev 37, e3439 (2021).
Lazar, H. L. et al. The Society of Thoracic Surgeons practice guideline series: Blood glucose management during adult cardiac surgery. Ann Thorac Surg 87, 663–669 (2009).
Rynazal, R. et al. Leveraging explainable AI for gut microbiome-based colorectal cancer classification. Genome Biol 24, 21 (2023).
Fenlon, C., O’Grady, L., Doherty, M. L. & Dunnion, J. A discussion of calibration techniques for evaluating binary and categorical predictive models. Prev Vet Med 149, 107–114 (2018).
Li, W. & Guo, Q. Plotting receiver operating characteristic and precision-recall curves from presence and background data. Ecol Evol 11, 10192–10206 (2021).
Seixas, A. A. et al. Associations of digital neuro-signatures with molecular and neuroimaging measures of brain resilience: The altoida large cohort study. Front Psychiatry 13, 899080 (2022).
Schaller, H. C. et al. On-line adaptive algorithm with glucose prediction capacity for subcutaneous closed loop control of glucose: evaluation under fasting conditions in patients with Type 1 diabetes. Diabet Med 23, 90–93 (2006).
Pappada, S. M. et al. An Artificial Neural Network-based Predictive Model to Support Optimization of Inpatient Glycemic Control. Diabetes Technol Ther 22, 383–394 (2020).
van den Boorn, M. et al. The development of a glucose prediction model in critically ill patients. Comput Methods Programs Biomed 206, 106105 (2021).
Hovorka, R. et al. Closing the loop: the adicol experience. Diabetes Technol Ther 6, 307–318 (2004).
Johnston, L. E. et al. Postoperative Hypoglycemia Is Associated With Worse Outcomes After Cardiac Operations. Ann Thorac Surg 103, 526–532 (2017).
Sanjay, O. P., Prashanth, P. & Tauro, D. I. Attempting to maintain normoglycemia during cardiopulmonary bypass with insulin may initiate post-operative hypoglycemia. Indian J Clin Biochem 18, 119–126 (2003).
Vongsumran, N., Buranapin, S. & Manosroi, W. Standardized Glycemic Management versus Conventional Glycemic Management and Postoperative Outcomes in Type 2 Diabetes Patients Undergoing Elective Surgery. Diabetes Metab Syndr Obes 13, 2593–2601 (2020).
Roth, J. et al. Blood Sugar Targets in Surgical Intensive Care—Management and Special Considerations in Patients With Diabetes. Dtsch Arztebl Int 118, 629–636 (2021).
Seo, D. H. et al. Status of Emergency Signal Functions in Myanmar Hospitals: A Cross-Sectional Survey. West J Emerg Med 20, 903–909 (2019).
Wang, J. et al. Association of lower body mass index with increased glycemic variability in patients with newly diagnosed type 2 diabetes: a cross-sectional study in China. Oncotarget 8, 73133–73143 (2017).
Knaak, C. et al. Risk Factors of Intraoperative Dysglycemia in Elderly Surgical Patients. Int J Med Sci 16, 665–674 (2019).
Cueni-Villoz, N. et al. Increased blood glucose variability during therapeutic hypothermia and outcome after cardiac arrest. Crit Care Med 39, 2225–2231 (2011).
Krinsley, J. S. Glycemic variability and mortality in critically ill patients: the impact of diabetes. J Diabetes Sci Technol 3, 1292–1301 (2009).
He, H. M. et al. Simultaneous assessment of stress hyperglycemia ratio and glycemic variability to predict mortality in patients with coronary artery disease: a retrospective cohort study from the MIMIC-IV database. Cardiovasc Diabetol 23, 61 (2024).
Chao, H. Y. et al. Association of In-Hospital Mortality and Dysglycemia in Septic Patients. PLoS One 12, e0170408 (2017).
Yue, S. et al. Machine learning for the prediction of acute kidney injury in patients with sepsis. J Transl Med 20, 215 (2022).
Deng, X., Li, M., Deng, S. & Wang, L. Hybrid gene selection approach using XGBoost and multi-objective genetic algorithm for cancer classification. Med Biol Eng Comput 60, 663–681 (2022).
Hou, N. et al. Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost. J Transl Med 18, 462 (2020).
Li, Y., Lyu, J., Wu, Y., Liu, Y. & Huang, G. PRIP: A Protein-RNA Interface Predictor Based on Semantics of Sequences. Life (Basel) 12, 307 (2022).
Wang, X. et al. A radiomics model combined with XGBoost may improve the accuracy of distinguishing between mediastinal cysts and tumors: a multicenter validation analysis. Ann Transl Med 9, 1737–1737 (2021).
Jiang, Y. Q. et al. Preoperative identification of microvascular invasion in hepatocellular carcinoma by XGBoost and deep learning. J Cancer Res Clin Oncol 147, 821–833 (2021).
Ma, B., Yan, G., Chai, B. & Hou, X. XGBLC: an improved survival prediction model based on XGBoost. Bioinformatics 38, 410–418 (2022).
Brugnara, G. et al. Multimodal Predictive Modeling of Endovascular Treatment Outcome for Acute Ischemic Stroke Using Machine-Learning. Stroke 51, 3541–3551 (2020).

No competing interests reported.

SupportingFile.zip

Download PDF

Reviewers agreed at journal
18 Nov, 2024
Reviewers invited by journal
17 Nov, 2024
Editor assigned by journal
08 Nov, 2024
Editor invited by journal
10 Aug, 2024
Submission checks completed at journal
10 Aug, 2024
First submitted to journal
02 Aug, 2024

You are reading this latest preprint version

Development and validation of machine learning models for glycemic variability in non-diabetic patients following cardiopulmonary bypass: a prospective observational study

Status:

Version 1

Abstract

Figures

1. Introduction

2. Methods

2.1 Study setting

2.2 Study population

2.3 Outcome measure

2.4 Predictor variables

2.5 Definition of the predictors

2.6 Glucose management

2.7 Missing data handling

2.8 Machine learning development process

2.9 Machine learning explainable tool

2.10 Statistical analysis

3. Results

3.1 Study population

3.2 Model performance comparisons

3.3 Explanation of the XGBoost Model via SHAP Approach

4. Discussion

5. Conclusions

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1