Nomogram construction of the difference in MEWS combined with SPO 2 for predicting the condition of patients with unplanned enter to the rescue room- A multi-centre study

doi:10.21203/rs.3.rs-4897287/v1

Purpose:

Investigate the efficacy of the difference in Modified Early Warning Score (MEWS) (within admission to the Emergency rescue Room versus 1 h of admission to the emergency observation room) combined with Pulse Oxygen Saturation(SPO₂)in early risk screening for unplanned transfer to the Emergency Rescue Room༈UTER༉.

Methods:

We collected the vital sign data of patients who are UTER in the emergency department of three hospitals and explored the predictive power of the difference in MEWS combined with the peripheral oxygen saturation (SPO₂) of patients when they are UTER by constructing a binary logistic regression model and nomogram. The prediction model is evaluated by ROC, AUC, calibration curve and other indicators.

Results:

The optimal cut-off value for predicting the difference in MEWS is 3.5, and the maximum value of Youden index is 0.761. When patients are unplanned to enter the resuscitation room, the first measured SPO₂ (P = 0.027, OR = 0.97, 95%CI: 0.94-1.00) and the difference in MEWS (P < 0.001, OR = 53.26, 95%CI: 29.51–96.11) are independent risk factors for predicting the progression of their condition. The training set predictive models AUC is 0.890 (95% CI: 0.853–0.927) and the internal and external validation sets’ predictive models AUC are 0.957 (95% CI: 0.917–0.997) and 0.934 (95%CI: 0.871–0.997), respectively.

Conclusions:

When a patient is UTER, the SPO2 measured for the first time and the difference in MEWS are independent risk factors for predicting the progression of their disease. The prediction model and the nomogram constructed by this can well predict the risk of disease change of the patient and improve the attention and vigilance of clinicians and nursing staff to the patient's disease.

Health sciences/Medical research/Study design

Health sciences/Medical research

Health sciences/Risk factors

The difference in MEWS

SPO2

nomogram

unplanned enter to resuscitation room

If the turnover of emergency detention patients is fast and no timed condition monitoring tools are available, leading to unstable conditions and potential risks, they may be transferred to the emergency room for improved consultation and treatment. This patient is defined as UTER patient. Therefore, early identification of critical signals in patients, accurate assessment, and timely prediction of their condition have become key focuses in emergency room nursing care[1, 2]. The MEWS was developed by Subbe et al. in 2001 as a modification of the Early Warning Score (EWS)[3]. Initially used in the intensive care unit (ICU), the MEWS have been employed across various healthcare settings, including hospital wards, emergency departments (EDs), and prehospital community settings. MEWS primarily aim to detect clinical deterioration in patients by monitoring vital signs, with high MEWS triggering a response to prevent any potential clinical decline. It is observed that patients' vital signs usually change before any clinical deterioration occurs[4–6]. Patients with high MEWS and low SPO₂ who are kept under observation are transferred to the rescue room to facilitate timely detection of changes in their condition, timely and aggressive treatment, and improvement in prognosis[7].

Monitoring various physiological parameters is essential for the early identification of patients at risk of deterioration during acute care stays. The most readily available parameters in such settings are the MEWS and SPO2. An elevated MEWS identifies patients at risk of deterioration, prompting the activation of a rapid response team[8]. However, the MEWS taken at the time of patient's stay and at the time of change in condition are the values of time nodes, which do not provide a dynamic early warning. Therefore, this study proposed using the difference in MEWS (The difference in MEWS is defined as the score upon unplanned admission to the rescue room minus the score upon entering initially the emergency department. The following is described as ΔMEWS) combined with SPO₂ (It was first measured by the moment when the patient was unplanned to enter the rescue room) to assess patients’ conditions in the emergency Department. After reviewing the literature^[9–11], we found that most of the research focused on the patients who return to the emergency room or the observation area, and there were few studies on the UTER. Based on this, this article also aims to explore the clinical and nursing guidance value of ΔMEWS and SPO₂ in predicting the disease progression of UTER, so as to better identify potentially critically ill patients, improve the quality of care, and ensure patient safety.

2.1 Study participants

This study included participants from the First Hospital of Jilin University, Hainan Provincial People's Hospital, and the Second Hospital of Shanxi Medical University. The inclusion criteria were as follows: (1) patients whose conditions were graded as level 3 or 4 at the time of observation; (2) patients who had a clear disease diagnosis and were under observation per the doctor’s order; and (3) patients who were transferred to the rescue room due to the changes in their condition while under observation in the emergency Department. The exclusion criterion was as follows: (1) patients with unclear diagnoses or missing data. This study was approved by the ethics committee of the First Hospital of Jilin University (2023 I KS I 136).The external validation data were also approved by the Ethics Committees of Hainan Provincial People's Hospital and the Second Hospital of Shanxi Medical University.

All methods were performed in accordance with the relevant guidelines and regulations（Data availability：The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.）

2.2 Data collection

The data were collected retrospectively through the emergency care systems of the three hospitals. The patient collected were all UTER patients, including their gender, age, major clinical diagnosis, and their MEWS within entering initially the emergency department and MEWS at the time of admission to the rescue room. The MEWS items mainly include respiratory rate, systolic blood pressure, heart rate, temperature, and state of consciousness. In addition, we recorded the SPO2 when they enter the rescue room. The assigned scores for each of the MEWS indexes are shown in Table 1.(Data availability:The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.)

Table 1. Assignment of values to the indicators of the MEWS score

Variables	MEWS Score
Variables	3	2	1	0	1	2	3
Temperature(℃)		≤35.0	35.0-36.1	36.1-38.0	38.1-38.5	＞38.5
Respiratory rate (bpm)		≤8		9-14	15-20	21-29	＞29
Heart rate(bpm)		≤40	41-45	51-100	101-110	111-130	＞130
SBP (mmHg)	≤70	71-80	81-100	101-199		＞200
AVPU Score				A	V	P	U

The total MEWS score ranges from 0 to 14, with higher scores indicating that the patient is more severely ill. SBP stands for systolic blood pressure；AVPU stands for different levels of consciousness：A is alert，V is verbal, P is pain, U is unresponsive.

We recorded the total score of the MEWS calculated within entering initially the emergency department of patients to the emergency department as M1. The total score of the MEWS is recorded as M2 when the patients enter the rescue room. The value of M2 minus M1 is what we defined earlier as ΔMEWS.The MEWS≥6 is categorized as high risk in the literature^[12]. Therefore, we define patients with A2≥6 as high-risk patients and A2<6 patients as low-risk patients.

Based on the emergency physician's primary diagnosis, the diseases were categorized into nine groups: the urinary system diseases (primary diagnoses include urinary stones, acute or chronic renal failure, uremia, pyelonephritis, and nephrotic syndrome), endocrine system diseases (primary diagnoses include diabetic ketoacidosis, hyperosmolar hyperglycemic coma, hypoglycemia, electrolyte disorders, dehydration, and septicemia), digestive system diseases (primary diagnoses include gastrointestinal bleeding, gallbladder stones, acute pancreatitis, acute obstructive septic cholecystitis, acute appendicitis, mesenteric vein thrombosis, and liver abscess), respiratory system diseases (primary diagnoses include bacterial or viral pneumonia, acute upper respiratory tract infections, spontaneous pneumothorax, bronchial asthma, respiratory failure types I and II, and bronchiectasis with hemoptysis), cardiovascular system diseases (primary diagnoses include heart failure, coronary atherosclerotic heart disease, acute myocardial infarction, unstable angina, acute pericardial effusion, cardiac arrhythmia, cardiac valvopathy, and hypertensive emergencies), neurological disorders (primary diagnoses include viral meningitis, low cranial pressure syndrome, epileptic seizures, transient ischemic attacks of the brain, cerebral hemorrhage, and cerebral infarction), poisonings, trauma and allergies(primary diagnoses include car accident injuries, carbon monoxide poisoning, electric shock, anaphylaxis, drug or food poisoning, snake venom poisoning, and knife stabbing), hematologic diseases (primary diagnoses include acute leukemia, hemophilia, hemolytic anemia, idiopathic thrombocytopenia, multiple myeloma, and leukocytosis), ENT (ear, nose, and throat) disorders, ocular diseases and neck disorders (primary diagnoses include acute pharyngitis, glaucoma, neck swelling, laryngeal swelling, maxillofacial cellulitis, and lens detachment).

2.3 Statistical analysis

R Studio (4.2.1) was used to statistically analyze the data, and binary logistic regression curves were used for univariate and multivariate tests. The measurement data were first subjected to the one-sample Kolmogorov-Smirnov normality test and Levene's chi-square test. The indicators met the criteria for normal distribution, and the chi-square test was expressed as mean ± standard deviation; the differences between the groups were compared using the independent-sample Student t test. The indicators that did not meet the criteria for normal distribution were expressed as the median and interquartile range [M (P₂₅,P₇₅ )]; the differences between groups were compared using the Mann-Whitney U test. Count data were compared using the nonparametric chi-square test. A P value <0.05 indicated a statistically significant difference.

According to the inclusion and exclusion criteria, 599 UTER patients in the emergency department in 2023 in the First Hospital of Jilin University were enrolled as a training set. Another 145 UTER patients in 2024 in the First Hospital of Jilin University were enrolled as the internal validation set (time independent). Further, 231 emergency department UTER patients from December 1, 2022, to March 1, 2023, in Hainan Provincial People's Hospital and from December 1, 2022, to March 1, 2023, in the Second Hospital of Shanxi Medical University were enrolled as the external validation set (geographically independent).

3.1 Optimal critical point selection for ΔMEWS

Considering the complexity of the condition of emergency patients and the unpredictable speed of disease progression, timely judgment on the significance of ΔMEWS is challenging for clinicians and nurses. Therefore, we try to find the optimal cut-off value of ΔMEWS, simplify the evaluation process, and enable faster judgment of the patient's condition. We first conduct a between-group comparison of ΔMEWS between patients with different risks of UTER patients and found that the P value was <0.001. We then construct alone a binary logistic regression model of ΔMEWS for patients with different UTER patients risks and plot the receiver-operating characteristic (ROC) curves. We calculate the maximum value of Youden index (where the model sensitivity and specificity were the highest) to determine the optimal cut-off value of the model. The results show that Youden index is maximal at 0.761, corresponding to a model sensitivity and specificity of 0.837 and 0.924, respectively. The corresponding ΔMEWS at this point is 3.5, as shown in Figure 1. Therefore, we define ΔMEWS≥3.5 as the high-alert group of diseases progression and ΔMEWS<3.5 as the low-alert group of diseases progression. This grouping is used to construct the nomogram and predictive models.

3.2 Binary logistic regression models for predicting the risk of the condition in UTRPs patients

The results of using binary logistic regression univariate and multivariate analyses for patients with different risks of UTER patients are shown in Table 2. SPO₂ (P = 0.027) and ΔMEWS (P < 0.001) in patients at UTER could be used as independent risk factors for predicting the progression of patients' disease and UTER patients. The specific results are shown in Table 2.

Table 2. Univariate versus multivariate analyses using binary logistic regression for patients with different risks of UTER patients

Items		Low-alert group (N=458)	High-alert Group (N=141)	Univariate analysis		Multifactorial analysis
				OR	P-value	OR	P-value
age		59.9 ± 16.5	61.3 ± 15.8	1.01 (0.99-1.02)	0.354
gender	female	186 (40.6%)	43 (30.5%)	Ref	-	Ref	-
	male	272 (59.4%)	98 (69.5%)	1.56 (1.04-2.33)	0.031	1.62(0.89-2.95)	0.121
urinary system diseases	No	454 (99.1%)	140 (99.3%)	Ref	-
	Yes	4 (0.9%)	1 (0.7%)	0.81 (0.09-7.31)	0.852
endocrine system diseases	No	443 (96.7%)	139 (98.6%)	Ref	-
	Yes	15 (3.3%)	2 (1.4%)	0.42 (0.10-1.88)	0.260
digestive system diseases	No	390 (85.2%)	134 (95%)	Ref	-	Ref	-
	Yes	68 (14.8%)	7 (5%)	0.30 (0.13-0.67)	0.003	0.58(0.19-1.76)	0.341
Respiratory system diseases	No	318 (69.4%)	109 (77.3%)	Ref	-
	Yes	140 (30.6%)	32 (22.7%)	0.67 (0.43-1.04)	0.072
cardiovascular system diseases	No	434 (94.8%)	134 (95%)	Ref	-
	Yes	24 (5.2%)	7 (5%)	0.94 (0.40-2.24)	0.897
neurological disorders	No	289 (63.1%)	57 (40.4%)	Ref	-	Ref	-
	Yes	169 (36.9%)	84 (59.6%)	2.52 (1.71-3.71)	<0.001	1.16(0.62-2.17)	0.638
Poisoning, trauma, allergies	No	438 (95.6%)	138 (97.9%)	Ref	-
	Yes	20 (4.4%)	3 (2.1%)	0.48 (0.14-1.63)	0.236
hematologic diseases	No	442 (96.5%)	136 (96.5%)	Ref	-
	Yes	16 (3.5%)	5 (3.5%)	1.02 (0.37-2.82)	0.976
ENT disorders, ocular diseases, neck disorders	No	456 (99.6%)	141 (100%)	Ref	-
	Yes	2 (0.4%)	0 (0%)	0 (0-Inf)	0.983
SPO₂		93.9 ± 6.8	85.7 ± 23.7	0.96 (0.94-0.97)	<0.001	0.97 (0.94-1.00)		0.027
ΔMEWS	<3.5	423 (92.4%)	23 (16.3%)
	≥3.5	35 (7.6%)	118 (83.7%)	62.00 (35.27-109.02)	<0.001	53.26 (29.51-96.11)		p<0.001

Our binary logistic regression model training set cohort constructed with SPO₂ and ΔMEWS had the following performance metrics: an accuracy of 0.903, a precision of 0.771, a sensitivity of 0.837, a specificity of 0.924, and an F1 value of 0.803. The AUC was 0.890 (95% CI:0.853-0.927) (Figure 2). We assess the model for multicollinearity using the Variance inflation factor (VIF) test, which show no covariance between SPO₂ and ΔMEWS (VIF=1.0011 for both). The model is evaluated using internal and external validation sets cohorts. We found that the accuracy of the internal and external validation sets cohorts are 0.924 and 0.944, precision are 0.930 and 0.55, sensitivity are 0.833 and 0.733, specificity are 0.969 and 0.958, F1 values are 0.879 and 0.629, AUC are 0.957 (95% CI: 0.917-0.997) and 0.934 (95% CI: 0.871-0.997), respectively. Risk

3.3 Evaluation of nomogram creation and predictive efficacy

Based on the constructed binary logistic regression model, a nomogram predictive risk map that allowed clinicians to make quick judgments about the patient's disease progression was plotted (Figure 3). This increased the clinicians' and caregivers' attention and vigilance to the patient's disease progression.

The nomogram is used to predict the risk probability of disease progression in patients with UTER patients, and when the risk is greater than 0.5, it can be considered that the patient has a high risk of disease progression and should be regarded as a high disease warning.

The calibration curve results are shown in Figure 4, and the Hosmer-Lemeshow test analysis (c²=8.19, P=0.316) showed a good fit for this training set. The H-L test was used for the internal validation set and the external validation set, and the results showed that the internal validation set c²=5.11, P=0.746 and the external validation set c²=3.08, P=0.687. Detailed calibration curves are available in the supplementary materials.

We further plotted the decision curve based on the model, and the decision curve of the training set model showed that there was a clinical net benefit rate using the model when the prediction threshold was in the range of 0.05-1.0 (Figure 5). Clinical decision curves for the internal and external validation sets can be found in the supplementary materials.

The decision analysis curve shows the net return on a series of threshold probabilities using and not using model diagnostics.

The clinical impact curves showed a high-level match between model predictions and actual occurrences and a high clinical prediction validity when the prediction threshold was greater than 0.6 (Figure 6). The calibration curves, decision curves, and clinical impact curves for the internal versus external validation sets are provided in the Supplementary Material.

4.1 Significance of constructing a nomogram of the risk of change in the condition of patients retained in the emergency department based on ΔMEWS combined with SPO ₂.

The condition and vital signs of patients in the emergency detention area are relatively stable, characterized by a large number, high mobility, and many types of diseases. Theoretically speaking, the condition of patients detained in the observation area seems to be stable, but the development of those who are hidden is easy to be ignored. Moreover, the patients who are detained in the observation area are not monitored in real time for vital signs. Hence, the potential changes in their condition cannot be easily found out, making the treatment difficult^{[13, 14]}. Consequently, timely and effective prediction of the risk of disease progression in these patients and appropriate interventions before transfer to the rescue room are crucial for the overall treatment of patients. Most studies have focused on unplanned return to the emergency room and changes in the condition of emergency patients, with few studies on risk prediction of emergency patients at the time of UTER ^[15]. Our study was novel in introducing a simple and easy-to-use method for predicting UTRPs risks. Our study found that ΔMEWS and SPO2 at UTER was an independent risk factor for predicting disease progression in patients who stayed in the emergency room. The study changed the nurses' prediction of the change in patient condition from empirical to scientific by applying objective indicators. The potential risk of patients undergoing emergency observation can be detected promptly by combining the difference between SPO2 at UTER and daily MEWS and the MEWS on the day of admission to the observation area. If the risk threshold is reached, the doctor is informed promptly and appropriate interventions are given, reflecting anticipatory nursing care and protecting the patients.

4.2 ΔMEWS combined with SPO ₂ at admission as an independent risk factor for progression of UTER in patients retained in the emergency department.

Previous studies found that most deteriorating patients already showed changes in their vital signs before the 6- to 24-h period preceding the catastrophic developments, such as unexpected ICU transfer or cardiopulmonary arrest^[16–18]. Therefore, various MEWS models based on physiological parameters have been suggested and applied^[19]. In this study, the first nomogram was constructed for predicting the progression of emergency patients in UTER using ΔMEWS combined with SPO2 based on the characteristics of the changes in patients in the emergency detention area. ΔMEWS and SPO2 in UTER can be used as independent risk factors for predicting the progression of patients' conditions. Different cut-off points of various EWS scores applied in the ED had comparable abilities to predict clinical decline, including death among hospitalized patients^[20]. The change in the patient's condition in the observation area is a dynamic process and, hence, cannot be accurately predicted using MEWS combined with SPO2. ΔMEWS combined with SPO2 dynamically respond to the change in patient's condition. The medical staff only need to enter the daily values into the system after calculating the difference. Looking at the column chart for calculating the degree of risk, the medical staff can intuitively find out the change in patient's condition and address it appropriately in time.

In this study, we constructed a nomogram for predicting UTER patients, which included ΔMEWS combined with SPO2. The validation results showed that the nomogram could accurately predict UTER risks for patients who stayed in the rescue room. Emergency care nurses only need to monitor patients' MEWS and SPO2 on a daily basis and enter the monitored values into the nomogram to determine the risk level of patients, which can scientifically improve the risk prediction ability of patients. The patients included in this study were from three hospitals in the north and south regions of China. Also, the internal validation was based on patients' information in different periods in the local hospitals, whereas the external validation was based on patients' information in the same period outside the region. Further, the nomogram was accurate and predictive.

Consent Statement:

Due to the retrospective nature of the study, (ethics committee of the First Hospital of Jilin University) waived the need of obtaining informed consent

Author contributions

F. C. and Q.Y.wrote the main manuscript text and F.Y. prepared figures 1-6 and tables 1-2.Y.Z. and H.provided external validation dataAll authors reviewed the manuscript.

The Competing Interests Statement

All the authors disclosed no relevant relationships

Data Availability

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

GROSSI A, RODRIGUEZ J E R, DE FREITAS SOUSA A A, et al. Management of unusual rectal foreign body - Case report and literature review [J]. Int J Surg Case Rep, 2022, 94: 107051.
TOY S, OZDEMIR F, KıZıLAY F, et al. Early rehabilitation results in a child who developed herpes simplex encephalitis [J]. North Clin Istanb, 2017, 4(3): 273-4.
CEI F, FENU P, SOLE C, et al. A quick modified early warning score for triaging medical patients at admission [J]. Eur J Emerg Med, 2022, 29(1): 80-1.
MARTíN-RODRíGUEZ F, LóPEZ-IZQUIERDO R, DEL POZO VEGAS C, et al. Predictive value of the prehospital NEWS2-L —National Early Warning Score 2 Lactate— for detecting early death after an emergency [J]. Emergencias, 2019, 31(3): 173-9.
GERRY S, BONNICI T, BIRKS J, et al. Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology [J]. Bmj, 2020, 369: m1501.
GAO H, MCDONNELL A, HARRISON D A, et al. Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward [J]. Intensive Care Med, 2007, 33(4): 667-79.
FLENADY T, DWYER T, APPLEGARTH J. Accurate respiratory rates count: So should you! [J]. Australas Emerg Nurs J, 2017, 20(1): 45-7.
DEVOE B, ROTH A, MAURER G, et al. Correlation of the predictive ability of early warning metrics and mortality for cardiac arrest patients receiving in-hospital Advanced Cardiovascular Life Support [J]. Heart Lung, 2016, 45(6): 497-502.
DELGADO-HURTADO J J, BERGER A, BANSAL A B. Emergency department Modified Early Warning Score association with admission, admission disposition, mortality, and length of stay [J]. J Community Hosp Intern Med Perspect, 2016, 6(2): 31456.
MAFTOOHIAN M, ASSARROUDI A, STEWART J J, et al. Evaluating the Use of a Modified Early Warning Score in Predicting Serious Adverse Events in Iranian Hospitalized Patients: A Prognostic Study [J]. J Emerg Nurs, 2020, 46(1): 72-82.
KNOERY C, BARLAS R S, VART P, et al. Modified early warning score and risk of mortality after acute stroke [J]. Clin Neurol Neurosurg, 2021, 202: 106547.
CONSTANTINESCU C, PASCA S, ILUTA S, et al. The Predictive Role of Modified Early Warning Score in 174 Hematological Patients at the Point of Transfer to the Intensive Care Unit [J]. J Clin Med, 2021, 10(20).
DURANTEZ-FERNáNDEZ C, POLONIO-LóPEZ B, MARTíN-CONTY J L, et al. Comparison of Nine Early Warning Scores for Identification of Short-Term Mortality in Acute Neurological Disease in Emergency Department [J]. J Pers Med, 2022, 12(4).
HUDSON P, EKHOLM J, JOHNSON M, et al. Early identification and management of the unstable adult patient in the emergency department [J]. J Clin Nurs, 2015, 24(21-22): 3138-46.
HINKELBEIN J, SPELTEN O, MARKS J, et al. An assessment of resuscitation quality in the television drama Emergency Room: guideline non-compliance and low-quality cardiopulmonary resuscitation lead to a favorable outcome? [J]. Resuscitation, 2014, 85(8): 1106-10.
SCHEIN R M, HAZDAY N, PENA M, et al. Clinical antecedents to in-hospital cardiopulmonary arrest [J]. Chest, 1990, 98(6): 1388-92.
FRANKLIN C, MATHEW J. Developing strategies to prevent inhospital cardiac arrest: analyzing responses of physicians and nurses in the hours before the event [J]. Crit Care Med, 1994, 22(2): 244-7.
HILLMAN K M, BRISTOW P J, CHEY T, et al. Duration of life-threatening antecedents prior to intensive care admission [J]. Intensive Care Med, 2002, 28(11): 1629-34.
KWACK W G, LEE D S, MIN H, et al. Evaluation of the SpO2/FiO2 ratio as a predictor of intensive care unit transfers in respiratory ward patients for whom the rapid response system has been activated [J]. PLoS One, 2018, 13(7): e0201632.
GUAN G, LEE C M Y, BEGG S, et al. The use of early warning system scores in prehospital and emergency department settings to predict clinical deterioration: A systematic review and meta-analysis [J]. PLoS One, 2022, 17(3): e0265559.

No competing interests reported.

Nomogram construction of the difference in MEWS combined with SPO 2 for predicting the condition of patients with unplanned enter to the rescue room- A multi-centre study

Status:

Version 1

Abstract

Purpose:

Methods:

Results:

Conclusions:

Figures

1. introduction

2. Materials and Methods

3. Results

4. Discussion

Declarations

Data Availability

References

Additional Declarations

Status:

Version 1