Development of prediction models and predictors analysis for axial neck pain in patients undergoing cervical laminoplasty based on machine learning

doi:10.21203/rs.3.rs-4873462/v1

Background

Axial neck pain (ANP) is one of the most common complications after cervical laminoplasty, leading to severe pain, disability and economic loss. By predicting patient outcomes pre-operatively, patients undergoing cervical laminoplasty can benefit from more accurate patient care strategies. However, predicting postoperative ANP is challenging. The aim of this study was to develop a machine learning model to predict at the individual level whether a patient experiences postoperative ANP and to reveal baseline predictors of persistent neck pain after laminoplasty.

Methods

This retrospective study includes 1982 patients. The population characteristics, clinical symptoms and signs, imaging features and preoperative scale of patients were retrospectively collected as input variables. The outcome measure was whether the patient achieved minimal clinically significant difference (MCID) in the visual analogue scale (VAS) score for postoperative ANP. Models were trained and optimized by process of machine learning (ML), including feature engineering, data pre-processing, and 8:2 training/validation-testing split of datasets. The feature-reduced model was established afterwards, and its performance and feature importance were evaluated through internal and external testing.

Results

Among the models generated by 45 features, XGBoost model yielded the highest AUROC of 0.7631 (95% CI, 0.7221–0.8051). Age, preoperative mJOA score, VAS score, SF36-body pain, SF36-mental health, SF36-role emotional, SF36-physiological function, lower limb weakness, and positive Hoffmann’ sign were selected as input features to build the feature-reduced model. In both internal and external testing of the feature-reduced models, model of Logistic_Regression algorithms reached the best performance, with AUROC of 0.9047 (95% CI, 0.8633–0.9406) for internal testing and 0.9200 (95% CI, 0.8678–0.9676) for external testing.

Conclusion

In this study, models for predicting the progress of postoperative ANP based on machine learning were established. The Logistic Regression model had a good ability to predict ANP progression of CSM patients and achieved best performance in a multicenter independent testing cohort. Feature importance analysis revealed key baseline predictors of postoperative ANP. This study proved that the potential of ML to predict the progress of ANP after cervical laminoplasty was significant, providing research basis for the training of machine learning models with larger samples and more features in the future.

Axial neck pain

Cervical spondylotic myelopathy

Cervical laminoplasty

Machine learning

Predictive models

Baseline predictors

Cervical spondylotic myelopathy (CSM) is an age-related degenerative disease and the most common cause of neurological dysfunction in the spinal cord worldwide.[1] Cervical laminoplasty (CLP) is an accepted surgical method for the treatment of CSM, which provides adequate posterior decompression while minimizing the impact on cervical stability.[2] Traditionally, it has been used to treat CSM and ossification of the posterior longitudinal ligament (OPLL), especially in cases involving multi-segmental lesions.[3]

Axial Neck Pain (ANP) is defined as pain limited to the neck and shoulder area. ANP is the most common postoperative complication of posterior cervical surgery, especially CLP.[3, 4] At present, the mechanism and influencing factors leading to ANP are still indeterminate. Previous studies showed that ANP was influenced by multiple preoperative risk factors like age, gender, preoperative neck and shoulder pain, stiffness and etc, but the results was controversial.[5] A systematic review[6] of 26 studies and 1297 patients showed that the incidence of ANP ranged from 5.2–61.5%. ANP is easy to persist for a long time, which seriously reduces patients’ quality of life and satisfaction of surgery after operation. With the change of medical mode, the expectation of surgical prognosis of CSM patients is not only limited to the improvement of neurological function, but also to the overall improvement of physical, psychological and social function. Therefore, it is of great clinical significance to accurately identify the high-risk groups of ANP after CLP and to identify the influencing factors of ANP.

Machine learning (ML) is a subset of artificial intelligence that enables algorithms or classifiers to learn large, complex datasets and produce useful predictive outputs. AI and ML are increasingly used in the field of spinal surgery, including diagnosis, treatment, postoperative prognosis and decision-support systems. Previous studies showed that machine learning had higher predictive ability and stability than traditional statistical methods.[7–9] The main advantage of ML-based clinical prediction models is the ability to handle nonlinear relationships between predictor variables and outcomes compared with typically linear regression techniques. ML algorithms could capture complex patterns and interactions that traditional models may ignore.[10–12] In addition, ML algorithms could identify the most important predictive features, which helps clinicians identify which factors are most relevant to the particular outcomes.[13, 14] Moreover, ML algorithm is better than traditional model to generalize new data, which can improve the applicability of the model.[15] Overall, these advantages could lead to better clinical decision making and have great significance for identifying people at high risk for ANP, improving patient care and outcomes.

This study aimed to develop ML models to predict whether ANP got worse after CLP, identifying and analyzing related predictive features of ANP. And the models were tested with multi-center data to evaluate the ability to support clinical decision making.

This study is a retrospective study and without human intervention in all process. We followed the Transparent Reporting of Multivariable Prediction Models for Individual Prognosis or Diagnosis (TRIPOD).[16]

2.1 Patient Population

The study was approved by Peking University Third Hospital Medical Science Research Ethics Committee. The data came from the Electronic Data Capture (EDC) System in Peking University Third Hospital Information Center. All the private information was masked. Each patient met the following inclusion criteria: 1) age ≥ 18; 2) diagnosed with CSM; 3) underwent CLP; 4) no prior cervical spine surgery. Exclusion criteria: 1) patients combined with cervical spondylotic radiculopathy (CSR) or cervical sympathetic; 2) patients with cervical tumor, active infection, rheumatoid arthritis, cervical trauma, and ankylosing spondylitis; 3) patients with missing data or invalid follow-up records.

2.2 Baseline Data and Outcomes

The baseline data for model training included population characteristics, clinical symptoms, physical signs, imaging features, and preoperative scale data

2.2.1 Population characteristics: age, gender, smoking history and drinking history.

2.2.2 Clinical symptoms: numbness (upper limbs, lower limbs or trunk), weakness (upper limbs or lower limbs), pain (upper limbs, lower limbs, neck and shoulder, trunk), Band-like sensation, Sensation of walking on cotton, difficulty in fine motor skill, hypoesthesia, autonomic symptoms, poor bowel control;

2.2.3 Physical signs: muscular atrophy, neck tenderness, muscle weakness (shoulder, upper limb or lower limb), abnormal reflex (abdominal, upper limb or lower limb), positive Hoffmann’s sign, positive Babinski’s sign, positive Eaton test, positive Spurling test;

2.2.4 Imaging features: Cervical spinal stenosis, defined as sagittal diameter of spinal canal/sagittal diameter of vertebral body (i.e. Pavlov ratio) < 0.75

2.2.5 Scale data: ANP was assessed by visual analogue scale (VAS) in this study. Besides, the modified Japanese Orthopedic Association score (mJOA) and the Short Form 36 (SF-36) quality of life scale were used to assess cervical spinal cord function and preoperative quality of life, respectively.

In this study, natural language processing (NLP) algorithm was applied to extract population characteristics and clinical symptom data from unstructured medical history records. Data of physical signs were selected according to the orthopaedic standardized medical records in our center. The patients were followed up 6 months after surgery, and their mJOA score and VAS score of ANP were recorded. The age and scale data were recorded as continuous variables, and the remaining data were recorded as binary variables.

The primary outcome measure of this study was whether patients showed progression of ANP after surgery in VAS score by the minimum clinically significant difference (MCID). According to previous studies [17, 18], the MCID of the VAS score for neck pain was 2.6 points.

2.3 Data pre-processing and feature engineering

When extracting data with the NLP algorithm, a blank would be left in the baseline dataset if there was no matching description in the medical history records. Therefore, we filled in all missing data from binary features with "normal" or "negative". Records of patients with other types of cervical spondylosis diagnosis, missing scale data, invalid follow-up records, or unplanned reoperation records were eliminated. The data was normalized to ensure that all features were fairly weighted and on the same scale. Each continuous variable, such as scale data, is placed on a scale of 0 to 1 using min-max standardization. And feature engineering is done by creating digitized variables for binary features.

2.4 Training Machine Learning Models and Performance Evaluation

The dataset was randomly divided, with 80% into training/validation set and 20% into the testing set. Commonly used ML models, including Logistic Regression (LR), Support Vector Machines (SVM), Random Forests (RF), XGBoost, and LightGBM, were trained using default parameters. The error caused by randomization is reduced by 5-fold cross-validation, and the optimal model was selected according to receiver operating characteristic curve (ROC curve) and area under curve (AUC) values. Models were optimized using the default hyper-parameters provided by the Sklearn module. All ML models were trained and tested using VScode™ and Sklearn modules.

In addition to evaluating the model using the ROC curve and its area under the curve (AUROC), we also selected other metrics for a more comprehensive evaluation of the models. Precision-Recall curves (PRC) provided a visualized evaluation of the models and were effective when dealing with unbalanced datasets because it could explore the balance between accuracy and recall for different thresholds. AUPRC is the weighted area under PRC. Balanced accuracy was used to deal with feature-unbalanced datasets in binary classification. It was defined as the average of the recall rates obtained on each category. Precision was defined as the ratio of correct information extracted to the number of all information extracted. Recall rate was defined as the ratio of correct information extracted to the number of correct information in the dataset. The Brier score is a statistical tool used to measure the accuracy of a prediction model. It measured the mean square error between the predicted probability and the actual probability.[19] The lower the Brier score, the higher the accuracy of the prediction model. By quantitatively using AUROC, AUPRC, balanced accuracy, weighted precision, weighted recall and Brier scores, we could obtain a more complete and comprehensive evaluation of the model.

2.5 Building and validation of a Feature-reduced Model

This study tried to reduce the number of features to improve the compatibility of the model across different healthcare systems. We calculated the feature importance of each models and further explored these features based on clinical experience and selected feature subsets to build feature-reduced ML models. We extracted the required features from previous dataset to train the feature-reduced models. Outcome measures and model evaluation were the same as above.

To further evaluate the applicability and extensibility of the feature-reduced models, patients undergoing CLP in our center in 2022–2023 were applied as the internal testing dataset, while patients from Peking Union Medical College Hospital and Beijing Hospital were used as external testing dataset. All patients met the above criteria.

3.1 Baseline Patients

After searching patients underwent CLP in our center between 2012 and 2022 on the EDC System, a total of 1982 patients met the criteria. Of the 1982 patients, 344 were excluded due to diagnosis of cervical radiculopathy or cervical sympathetic neuropathy. 156 cases were excluded due to unplanned reoperation. 694 patients had missing data in their records and 123 had invalid data in their records. In the end, a dataset of 665 patients and 45 features was generated (Figure 1). The baseline features were shown in Table 1.

3.2 Model Performance

Five ML models (LightGBM, LR, RF, SVM, XGBoost) were generated using the training set. The ROC curve and AUROC value generated by 5-fold cross-validation algorithms were shown in Figure 2. Among the five ML models, SVM had the lowest AUROC of 0.7157 (95% CI, 0.6737-0.7566), while the decision tree algorithms (LightGBM, RF and XGBoost) showed better performance. The AUROC of XGBoost model was the highest, which was 0.7631 (95% CI, 0.7221-0.8051).

In addition, this study used metrics including balanced accuracy, weighted precision, weighted recall, weighted AUPRC and Brier score. The results were shown in Table 2. When evaluating the performance of models, we need to understand the complexity associated with unbalanced datasets in ML classification tasks, especially in datasets with a limited number of positive examples, which is common in real-world medical datasets. These metrics could evaluate the performance of models across categories and provide a comprehensive view of class distribution. In contrast, unweighted versions of these metrics may not provide reliable results in the case of unbalanced datasets because they ignored the category distribution of datasets and may mistakenly convey satisfactory performance by ignoring a few categories.

Because of its unique baseline value, the interpretation of AUPRC may be more special than other metrics such as AUROC. The baseline used by AUROC is 0.5, which represented the performance of random classifiers. On the contrary, the baseline of AUPRC was determined by the proportion of positive samples in the dataset.[20] This difference may cause the value of AUPRC to be significantly lower than that of AUROC. For example, in this study, the weighted AUPRC of the XCBoost model is 0.4977 (95%CI, 0.4473-0.5481), while the proportion of patients with postoperative ANP progression in the dataset is 0.25, representing the baseline.

3.3 Feature Importance and Feature-reduced Model

This study calculated the feature importance of the five models, and the related results were shown in figure 4. In the evaluation of feature importance of five models, age, preoperative JOA score, preoperative VAS of neck and shoulder pain, SF36-BP, SF36-MH, SF36-PF and lower limb weakness were all at top rank. Considering the feature importance of each models, as well as the accessibility and compatibility of input data, in addition to the above features, we also chose SF36-RE, VAS of upper limb pain and positive Hoffmann’s sign as input features to build the feature-reduced model (a total of 10 input features). The differences of related features among people with different ANP outcomes were shown in Table 3.

We used the same training set to build feature-reduced models. Similar methods were used to optimize and test the models. The ROC curve was shown in figure 5 and the relevant model evaluation metrics were shown in Table 4. Except the SVM model (AUROC=0.5354, 95%CI, 0.4877-0.5841), decision tree algorithms (LightGBM, RF and XGBoost) and LR model all showed good performance.

3.4 Internal and External Validation

Internal testing dataset included 48 cases while external testing dataset included 23 cases in this study. Datas were extracted according to the input features of the feature-reduced model, and the relevant baseline features were shown in Table 5. The previously trained feature-reduced models were used to classify the testing datasets, and the ROC curve was shown in figure 6. In internal testing and external testing, RF, XGBoost and LightGBM could all be classified as good discriminatory ability (AUROC > 0.8). The model with the best performance was the LR model, whose AUROC values of internal testing and external testing reached 0.9047 (95% CI, 0.8633-0.9406) and 0.9200 (95% CI, 0.8678-0.9676), respectively, and other performance evaluation metrics were also outstanding (figure 6). It indicated that the models also performed well in extrapolation besides SVM. Table 4 showed the details of these performance evaluation metrics.

4.1 Application of ML in spine surgery

As a new technology, ML has been well proved in medical diagnosis and prognosis evaluation. [21] ML algorithms could predict output by given inputs like a simple regression model. [22] However, the statistical knowledge used in ML is more complex when generating predictions from input data. In terms of spine, ML has great potential with many advantages, including the ability to deal with large datasets and capture nonlinear relationships compared with traditional statistical models. In 2019, Fehlings et al[23] developed a ML model to predict the surgical outcome of DCM patients based on the improvement of SF-6D quality of life score and mJOA score. 539 patients completed 2-year follow-up, and the best performance prediction model used a random forest structure, with an average AUC of 0.70. In 2021, Wang et al[24] used an artificial neural network model to stratify the ACDF risk of 12492 patients from the National Surgical quality improvement Program database. If patients had any complications within a week after the first operation, they would be considered "unsafe" out-patient surgery. The AUC of the model was 0.740, which was significantly higher than the AUC of American Society of Anesthesiologists (P < 0.05). DiSilvestro et al [25] used a Bayesian classification algorithm to predict 30-day mortality after spinal tumor resection in the National Surgical Quality Initiative Program. The algorithm exceeded the predictive ability of the National Surgical Quality Initiative's Probability of Death calculator. And its accuracy continued to improve as the model continued to learn from input patient data. Similarly, Karhade et al [9] used 1790 patients to evaluate the effectiveness of several ML models in predicting 30-day mortality after surgery for spinal metastasis and integrated them into an open-access Web application. Bayesian classification algorithm had the best result in terms of identification, calibration and overall performance, with an average AUC of 0.782. As the volume of data continues to grow, building learning systems and deploying them as accessible tools can greatly enhance the ability of ML model.

In summary, the advantages of ML could help clinicians improve medical level, improve work efficiency and reduce the occurrence of adverse events. However, more randomized controlled trials and improvement of interpretability are essential for clinicians to accept the help of ML models in practice.

4.2 Outcome measure and evaluation of ANP

This study used ML methods to develop five models to predict the progression of postoperative ANP. To our knowledge, the acute neck pain in the early stage after cervical spine surgery is mostly related to the surgical wound itself. In the early tissue healing process, the acute pain has the significance of warning and protection for the body. However, some postoperative acute neck pain can become chronic and persistent, and this part of the symptoms is defined as postoperative ANP. In order to identify patients with postoperative ANP more accurately and exclude the interference of factors such as acute pain, we chose to follow up patients at 6 months after surgery. Furthermore, because 1) the VAS score is completely assessed by the patient; 2) based on individual differences, each person has different tolerance to pain; 3) 21–38% of patients with CSM have moderate to severe neck pain before surgery.[26–28] Therefore, the simple evaluation of postoperative ANP severity (such as whether the VAS score is > 3 points) has limited reference significance. In order to better identify whether patients' ANP became worse after surgery, the outcome measure of ML model in this study was set as whether patients' ANP progress after surgery was greater than 2.6 points, which is the MCID in neck VAS score.

4.3 Feature importance and risk factors of ANP

In the test of the models, this study found that the RF, XGBoost and LightGBM models could well predict the postoperative ANP progress of patients, and the model with the best performance was the LR model. We calculated the feature importance of five models respectively, and selected the preoperative features to build feature-reduced models. As shown in Table 3, except for age and SF36-PF, there were significant differences in preoperative VAS score, preoperative mJOA score, SF36-MH, SF36-BP, SF36-RE, lower limb weakness and positive Hoffmann’s sign among patients with different ANP outcomes. All this features could be regarded as risk factors of post-operative ANP.

This study demonstrated that patients with lower preoperative VAS of neck and shoulder pain were more likely to experience ANP progression beyond MCID after surgery. In other words, patients with less ANP before surgery were more likely to experience the progression of ANP after surgery. This may because of: 1) the ceiling effect, progress of ANP after surgery in patients with high preoperative VAS score (e.g. VAS score 8–10) was difficult to be reflected by VAS score; 2) The subjectivity and sensitivity of pain evaluation. Patients with lower preoperative VAS scores were more sensitive to any possible postoperative ANP changes. Although previous studies [29, 30] showed that severe baseline neck pain led to moderate to severe post-operative ANP, this was not inconsistent with our study, because we focused more on the progression of ANP.

Through further analysis of patients with positive outcomes, we found that postoperative ANP progression greatly affected patients' quality of life, which was specifically manifested in low mJOA score and poor SF36-PF score, which would further affect patients' satisfaction of surgery. Similarly, Sherrod et al [27] proved that although patients in the ANP group were overall satisfied with the operation, the satisfaction rate was significantly lower than that in the group without ANP. Therefore, in the current era of value-based health care, it has great significance to re-understand which patients are likely to develop ANP after surgery, which can help guide risk stratification in clinical practice, enhancing patient informed consent and reducing drug consumption.[31]

In this study, the risk factors for postoperative ANP in CSM patients screened by ML methods were similar to those screened by traditional methods. It is widely believed that mental health conditions affect pain and function after surgery for many musculoskeletal diseases.[32] Trief et al[33] and Carreon et al[34] demonstrated that SF36-MH predicted surgical outcomes for lumbar degenerative diseases. Kimura et al[30] showed that a lower baseline SF-36 MCS score was significantly associated with worse axial symptoms after CLP, similar to the findings of this study.

In terms of clinical symptoms and signs, preoperative lower limb weakness and positive Hoffmann’s sign were more likely to lead to postoperative ANP progression. Previous studies [35–37] revealed that preoperative features such as lower limb numbness, weakness, and positive pathological signs were associated with poor overall neurological function improvement after surgery. However, how these factors affected the exacerbation of ANP remained to be further studied. Finally, it had been reported that elderly patients were more likely to develop ANP after surgery, which may be due to the poor tolerance of patients to surgery and the weakened recovery of postoperative muscle function with the increase of age.[26] This study found a similar trend, but the results showed not significant difference.

4.4 Advantages and limitatons

This study showed some advantages compared with previous studies. First, we applied NLP algorithms data extraction. The characteristics used in the initial models of this study included population characteristics, clinical symptoms, physical signs, imaging features, and preoperative scale data, which were more comprehensive than previous studies. Then, we could model complex non-linear relationships, avoid overfitting, and better predict individual CSM patients by using ML methods. Therefore, our models has better predictive performance than previously published models. Besides, our models were optimized by feature reduction, which was more convenient to serve clinical practice and reduces the complexity of usage. In addition, our models performed well in an independent patient cohort that was not used for model training, which indicated that our model had high generalizability. After incorporated into the CDSS system of our center, the performance of the model will be further improved with the accumulation of data.

Our study still had some limitations. (1) Compared with the analysis of multiple perioperative time points, our study only included preoperative and postoperative short-term follow-up data of patients, which might have limitations in describing the duration of symptoms and recovery trajectory of patients. (2) Previous studies showed that imaging features such as ectopic ossification, cervical kyphosis and cervical spondylolisthesis were correlated with postoperative ANP, but this study only included preoperative X-ray spinal stenosis as a feature in model training, which might have a certain impact on the effectiveness of the model. (3) The sample size of this study was limited. While ML provides a powerful way to build complex models and generate predictions, compared to traditional statistical methods, ML models require relatively large datasets to achieve optimal performance. Despite the above limitations, the results of this study preliminarily verified the feasibility of applying ML methods to postoperative ANP studies of CSM patients, and laid a foundation ML model training involving larger samples and more factors in the future.

This study retrospectively analyzed the postoperative ANP progression in CSM patients, and established prediction models for postoperative ANP progression based on ML. The LR model had good predictive ability for postoperative ANP progression in patients with CSM and had good accuracy in a multi-center independent testing cohort. Feature importance analysis showed that preoperative VAS score, mJOA score, SF36-MH, SF36- BP, SF36-RE, lower limb weakness and positive Hoffmann’ sign were the key predictive features, which were close to clinical experience. In conclusion, our analysis demonstrated the applicability of ML in predictive modeling of ANP-related outcomes after CLP. ML models could help identify patients who benefit from CLP rather than experiencing new post-operative pain exacerbations, which is important to spinal surgeons.

ANP

axial neck pain

MCID

minimal clinically significant difference

VAS

visual analogue scale

CLP

cervical laminoplasty

OPLL

ossification of the posterior longitudinal ligament

ML

machine learning

mJOA score

modified Japanese Orthopedic Association score

NLP

natural language processing

LR

Logistic Regression

SVM

Support Vector Machines

RF

Random Forests

AUROC

area under receiver operating characteristic curve

AUPRC

area under Precision-Recall curve

UL

upper limbs

LL

lower limbs

NSP

neck and shoulder pain

ULP

upper limbs pain

PF

physical Functioning

RP

Role-Physical

BP

Bodily Pain

GH

General Health

VT

Vitality

SF

Social Functioning

RE

Role-Emotional

MH

Mental Health

HT

Health Transition

7.1 Ethics approval and consent to participate

This study was approved by the Ethics Committee of Peking University Third Hospital (2021-160-02). Informed consent was obtained from all subjects in the database. All analysis was performed in accordance with relevant regulations of the committee and the Declaration of Helsinki.

7.2 Consent for publication

Not applicable

7.3 Availability of data and materials

The data used in this study are available in Electronic Data Capture (EDC) System in Peking University Third Hospital Information Center, but restrictions apply to public availability of these data used under license for the current study. Reasonable request for access to the database could be made by contacting the corresponding author for detailed process.

7.4 Competing interests

The authors declare that they have no competing interests.

7.5 Funding

None

7.6 Authors' contributions

XF and FZ designed the study; XF and SZ collected the data; LL analyzed the data; SZ and FZ supervised the study; XF wrote the manuscript.

7.7 Acknowledgements

Not applicable

Karadimas SK, Erwin WM, Ely CG, Dettori JR, Fehlings MG. Pathophysiology and natural history of cervical spondylotic myelopathy. Spine (Phila Pa 1976). 2013;38(22 Suppl 1):S21–36.
Fehlings MG, Wilson JR, Kopjar B, Yoon ST, Arnold PM, Massicotte EM, et al. Efficacy and safety of surgical decompression in patients with cervical spondylotic myelopathy: results of the AOSpine North America prospective multi-center study. J Bone Joint Surg Am. 2013;95(18):1651–8.
Qu L, Li Z, Wang X, Yuan L, Li C. Axial Symptoms After Conventional and Modified Laminoplasty: A Meta-analysis. World Neurosurg. 2023;180:112–22.
Hosono N, Yonenobu K, Ono K. Neck and shoulder pain after laminoplasty. A noticeable complication. Spine (Phila Pa 1976). 1996;21(17):1969–73.
Chen H, Liu H, Deng Y, Gong Q, Li T, Song Y. Multivariate Analysis of Factors Associated With Axial Symptoms in Unilateral Expansive Open-Door Cervical Laminoplasty With Miniplate Fixation. Med (Baltim). 2016;95(2):e2292.
Wang SJ, Jiang SD, Jiang LS, Dai LY. Axial pain after posterior cervical spine surgery: a systematic review. Eur Spine J. 2011;20(2):185–94.
Suri A, Jones BC, Ng G, Anabaraonye N, Beyrer P, Domi A, et al. A deep learning system for automated, multi-modality 2D segmentation of vertebral bodies and intervertebral discs. Bone. 2021;149:115972.
Edström E, Burström G, Nachabe R, Gerdhem P, Elmi Terander AA. Novel Augmented-Reality-Based Surgical Navigation System for Spine Surgery in a Hybrid Operating Room: Design, Workflow, and Clinical Applications. Oper Neurosurg (Hagerstown). 2020;18(5):496–502.
Karhade AV, Thio Q, Ogink PT, Shah AA, Bono CM, Oh KS, et al. Development of Machine Learning Algorithms for Prediction of 30-Day Mortality After Surgery for Spinal Metastasis. Neurosurgery. 2019;85(1):E83–91.
Chen JH, Asch SM. Machine Learning and Prediction in Medicine - Beyond the Peak of Inflated Expectations. N Engl J Med. 2017;376(26):2507–9.
Ghassemi M, Naumann T, Schulam P, Beam AL, Chen IY, Ranganath R. A Review of Challenges and Opportunities in Machine Learning for Health. AMIA Jt Summits Transl Sci Proc. 2020;2020:191–200.
Bzdok D, Krzywinski M, Altman N. Points of Significance: Machine learning: a primer. Nat Methods. 2017;14(12):1119–20.
Musolf AM, Holzinger ER, Malley JD, Bailey-Wilson JE. What makes a good prediction? Feature importance and beginning to open the black box of machine learning in genetics. Hum Genet. 2022;141(9):1515–28.
Rajkomar A, Dean J, Kohane I. Machine Learning in Medicine. N Engl J Med. 2019;380(14):1347–58.
Buddhiraju A, Chen TL, Subih MA, Seo HH, Esposito JG, Kwon YM. Validation and Generalizability of Machine Learning Models for the Prediction of Discharge Disposition Following Revision Total Knee Arthroplasty. J Arthroplasty. 2023;38(6s):S253–8.
Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350(jan07 4):g7594–g.
Chan AK, Shaffrey CI, Gottfried ON, Park C, Than KD, Bisson EF, et al. Cervical spondylotic myelopathy with severe axial neck pain: is anterior or posterior approach better? J Neurosurgery: Spine. 2023;38(1):42–55.
Jenkins NW, Parrish JM, Lynch CP, Cha EDK, Mohan S, Geoghegan CE, et al. The Association of Preoperative Duration of Symptoms With Clinical Outcomes and Minimal Clinically Important Difference Following Anterior Cervical Discectomy and Fusion. Clin Spine Surg. 2020;33(9):378–81.
Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17(1):230.
Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE. 2015;10(3):e0118432.
Yasaka K, Abe O. Deep learning and artificial intelligence in radiology: Current applications and future directions. PLoS Med. 2018;15(11):e1002707.
Khan O, Badhiwala JH, Wilson JRF, Jiang F, Martin AR, Fehlings MG. Predictive Modeling of Outcomes After Traumatic and Nontraumatic Spinal Cord Injury Using Machine Learning: Review of Current Progress and Future Directions. Neurospine. 2019;16(4):678–85.
Merali ZG, Witiw CD, Badhiwala JH, Wilson JR, Fehlings MG. Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy. PLoS ONE. 2019;14(4):e0215133.
Wang KY, Suresh KV, Puvanesarajah V, Raad M, Margalit A, Jain A. Using Predictive Modeling and Machine Learning to Identify Patients Appropriate for Outpatient Anterior Cervical Fusion and Discectomy. Spine. 2021;46(10):665–70.
DiSilvestro KJ, Veeramani A, McDonald CL, Zhang AS, Kuris EO, Durand WM, et al. Predicting Postoperative Mortality After Metastatic Intraspinal Neoplasm Excision: Development of a Machine-Learning Approach. World Neurosurg. 2021;146:e917–24.
Yoshida M, Tamaki T, Kawakami M, Nakatani N, Ando M, Yamada H, et al. Does reconstruction of posterior ligamentous complex with extensor musculature decrease axial symptoms after cervical laminoplasty? Spine (Phila Pa 1976). 2002;27(13):1414–8.
Sherrod BA, Michalopoulos GD, Mulvaney G, Agarwal N, Chan AK, Asher AL, et al. Development of new postoperative neck pain at 12 and 24 months after surgery for cervical spondylotic myelopathy: a Quality Outcomes Database study. J Neurosurgery: Spine. 2023;38(3):357–65.
Devin CJ, Asher AL, Alvi MA, Yolcu YU, Kerezoudis P, Shaffrey CI, et al. Impact of predominant symptom location among patients undergoing cervical spine surgery on 12-month outcomes: an analysis from the Quality Outcomes Database. J Neurosurg Spine. 2021;35(4):399–409.
Cook CE, George SZ, Asher AL, Bisson EF, Buchholz AL, Bydon M, et al. High-impact chronic pain transition in surgical recipients with cervical spondylotic myelopathy. J Neurosurg Spine. 2022;37(1):31–40.
Kimura A, Shiraishi Y, Inoue H, Endo T, Takeshita K. Predictors of Persistent Axial Neck Pain After Cervical Laminoplasty. Spine (Phila Pa 1976). 2018;43(1):10–5.
André A, Peyrou B, Carpentier A, Vignaux JJ. Feasibility and Assessment of a Machine Learning-Based Predictive Model of Outcome After Lumbar Decompression Surgery. Global Spine J. 2022;12(5):894–908.
Linton SJ. A review of psychological risk factors in back and neck pain. Spine (Phila Pa 1976). 2000;25(9):1148–56.
Trief PM, Ploutz-Snyder R, Fredrickson BE. Emotional health predicts pain and function after fusion: a prospective multicenter study. Spine (Phila Pa 1976). 2006;31(7):823–30.
Carreon LY, Djurasovic M, Dimar JR 2nd, Owens RK 2nd, Crawford CH 3rd, Puno RM, et al. Can the anxiety domain of EQ-5D and mental health items from SF-36 help predict outcomes after surgery for lumbar degenerative disorders? J Neurosurg Spine. 2016;25(3):352–6.
Badhiwala JH, Ahuja CS, Akbar MA, Witiw CD, Nassiri F, Furlan JC, et al. Degenerative cervical myelopathy - update and future directions. Nat Rev Neurol. 2020;16(2):108–24.
Tetreault L, Kopjar B, Côté P, Arnold P, Fehlings MG. A Clinical Prediction Rule for Functional Outcomes in Patients Undergoing Surgery for Degenerative Cervical Myelopathy: Analysis of an International Prospective Multicenter Data Set of 757 Subjects. J Bone Joint Surg Am. 2015;97(24):2038–46.
Tetreault LA, Kopjar B, Vaccaro A, Yoon ST, Arnold PM, Massicotte EM, et al. A clinical prediction model to determine outcomes in patients with cervical spondylotic myelopathy undergoing surgical treatment: data from the prospective, multi-center AOSpine North America study. J Bone Joint Surg Am. 2013;95(18):1659–66.

Table 1: Baseline features of the patients.

Variables	Total Mean (±SD), Median (IQR), or n (%)
Age	51.62 (11.05)
Gender
Male	356 (53.5%)
Female	309 (46.5%)
Smoking history
No	564 (84.8%)
Yes	101 (15.2%)
Drinking history
No	613 (92.2%)
Yes	52 (7.8%)
Clinical symptoms
Numbness in UL
No	166 (25.0%)
Yes	499 (75.0%)
Numbness in LL
No	490 (73.7%)
Yes	175 (26.3%)
Trunk Numbness
No	644 (96.8%)
Yes	21 (3.2%)
Weakness in UL
No	549 (82.6%)
Yes	116 (17.4%)
Weakness in LL
No	404 (60.8%)
Yes	261 (39.2%)
Pain in UL
No	492 (74.0%)
Yes	173 (26.0%)
Pain in LL
No	640 (96.2%)
Yes	25 (3.8%)
Neck pain
No	378 (56.8%)
Yes	287 (43.2%)
Shoulder pain
No	511 (76.8%)
Yes	154 (23.2%)
Trunk pain
No	582 (87.5%)
Yes	83 (12.5%)
Band-like sensation
No	625 (94.0%)
Yes	40 (6.0%)
Sensation of walking on cotton
No	395 (59.4%)
Yes	270 (40.6%)
difficulty in fine motor skill
No	573 (86.2%)
Yes	92 (13.8%)
hypoesthesia
No	659 (99.1%)
Yes	6 (0.9%)
autonomic symptoms
No	529 (79.5%)
Yes	136 (20.5%)
poor bowel control
No	656 (98.6%)
Yes	9 (1.4%)
Physical signs
muscular atrophy
No	600 (90.2%)
Yes	65 (9.8%)
neck tenderness
No	267 (40.2%)
Yes	398 (59.8%)
Decline in shoulder muscle strength
No	565 (85.0%)
Yes	100 (15.0%)
Decline in UL muscle strength
No	364 (54.7%)
Yes	301 (45.3%)
Decline in LL muscle strength
No	543 (81.7%)
Yes	122 (18.3%)
Abnormal Abdominal reflex
No	649 (97.6%)
Yes	16 (2.4%)
Abnormal UL reflex
No	642 (96.5%)
Yes	23 (3.5%)
Abnormal LL reflex
No	648 (97.4%)
Yes	17 (2.6%)
positive Hoffmann’s sign
No	180 (27.1%)
Yes	485 (72.9%)
positive Babinski’s sign
No	474 (71.3%)
Yes	191 (28.7%)
positive Eaton test
No	498 (74.9%)
Yes	167 (25.1%)
positive Spurling test
No	537 (80.8%)
Yes	128 (19.2%)
Imaging feature
Cervical spinal stenosis
No	395 (59.4%)
Yes	270 (40.6%)
Scale data
Pre-operative
VAS of NSP	2.90 (±2.59)
VAS of ULP	2.19 (±2.71)
mJOA score	13.34 (±2.84)
SF36-PF	68.70 (±24.94)
SF36-RP	19.95 (±36.34)
SF36-BP	68.70 (±24.94)
SF36-GH	54.52 (±26.71)
SF36-VT	62.28 (±27.14)
SF36-SF	62.72 (±26.11)
SF36-RE	27.83 (±42.29)
SF36-MH	67.72 (±22.49)
SF36-HT	4.25 (±0.74)
Post-operative
VAS of ANP	3.00 (±2.39)
mJOA score	15.22 (±1.79)
Change in neck pain
ΔVAS ≥ 2.6	166 (25.0%)
ΔVAS < 2.6	499 (75.0%)

UL, upper limbs; LL, lower limbs; NSP, neck and shoulder pain; ULP, upper limbs pain; PF, physical Functioning; RP, Role-Physical; BP, Bodily Pain; GH, General Health; VT, Vitality; SF, Social Functioning; RE, Role-Emotional; MH, Mental Health; HT, Health Transition.

Table 2: Performance metrics of the models generated by 45 features

Algorithm	Weighted precision (95% CI)	Weighted recall (95% CI)	Weighted AUPRC (95% CI)	Balanced accuracy (95% CI)	AUROC (95% CI)	Brier score (95% CI)
LightGBM	0.7093(0.6611-0.7575)	0.7319(0.6903-0.7734)	0.4540 (0.3698-0.5382)	0.5947 (0.5331-0.6562)	0.7443 (0.7030-0.7854)	0.1706 (0.1489-0.1923)
Logistic regression	0.7241 (0.6736-0.7746)	0.7439 (0.7011-0.7868)	0.4440 (0.3763-0.5118)	0.6124 (0.5588-0.6660)	0.7357 (0.6924-0.7779)	0.1744 (0.1560-0.1927)
Random forest	0.7036 (0.6490-0.7582)	0.7470 (0.7132-0.7808)	0.4764 (0.3933-0.5596)	0.5625 (0.5136-0.6114)	0.7536 (0.7117-0.7921)	0.1603 (0.1495-0.1710)
svm	0.6301 (0.5700-0.6902)	0.7259 (0.7014-0.7503)	0.4126 (0.3644-0.4607)	0.5059 (0.4827-0.5292)	0.7157 (0.6737-0.7566)	0.1709 (0.1618-0.1800)
XGboost	0.7224 (0.6953-0.7495)	0.7470 (0.7228-0.7712)	0.4977 (0.4473-0.5481)	0.6067 (0.5684-0.6451)	0.7631 (0.7221-0.8051)	0.1568 (0.1458-0.1679)

Table 3: Difference of features used for building Feature-reduced Models in the baseline dataset

Variables	Group 1 ΔVAS ANP ≥ 2.6	Group 2 ΔVAS ANP < 2.6	P value
Sample size	166	499
Age	52.14 (±11.27)	51.45 (±10.98)	0.428
Weakness of LL n（%）	85 (51.2%)	176 (35.3%)	<0.001
positive Hoffmann’s sign n（%）	139 (83.7%)	346 (69.3%)	<0.001
*Pre-operative scale（*Mean±SD**）
mJOA score	12.99 (±2.77)	13.45 (±2.85)	0.011
VAS of NSP	1.00 (±1.49)	3.55 (±2.56)	<0.001
VAS of ULP	1.30 (±2.20)	2.49 (±2.79)	<0.001
SF36-BP	70.33 (±24.05)	59.66 (±25.73)	<0.001
SF36-PF	67.14 (±24.94)	69.22 (±24.95)	0.273
SF36-MH	68.65(±21.99)	77.42 (±22.66)	0.047
SF36-RE	29.97 (±43.41)	37.12 (±41.94)	0.041
Post-operative mJOAscore	14.59 (±1.85)	15.42 (±1.73)	<0.001

For continuous variables such as age and score data, they were expressed as mean ±standard deviation; Shapiro-Wilk test was used to evaluate whether the data were normally distributed. For the data with normal distribution, one-way ANOVA was used to test the differences between the groups. For data with non-normal distribution, the rank sum test (Kruskal-Wallis test) was used to analyze the differences between groups. For the other classification variables, the expected frequency was calculated first. For the data with expected frequency ≥5, Pearson Chi-square test (χ2) was used to analyze the differences among all groups. For data with expected frequency <5, Fisher's exact test was used to analyze the differences among groups.

Table 4: Performance metrics of the Feature-reduced Models

Outcome	Algorithm	Weighted precision (95% CI)	Weighted recall (95% CI)	Weighted AUPRC (95% CI)	Balanced accuracy (95% CI)	AUROC (95% CI)	Brier score (95% CI)
Validation	LightGBM	0.7120(0.6815-0.7425)	0.7394(0.7139-0.7650)	0.4925(0.4277-0.5573)	0.5956(0.5568-0.6345)	0.7630(0.7233-0.791)	0.1646(0.1486-0.1806)
	Logistic regression	0.6929(0.6770-0.7088)	0.7184(0.6938-0.7429)	0.4279(0.3862-0.4696)	0.5757(0.5527-0.5987)	0.7637(0.7220-0.8032)	0.1570(0.1435-0.1705)
	Random forest	0.7201(0.6824-0.7578)	0.7417(0.7040-0.7795)	0.4920(0.4347-0.5493)	0.6034(0.5552-0.6515)	0.7614(0.7217-0.7991)	0.1603(0.1481-0.1725)
	svm	0.5614(0.5561-0.5667)	0.7492(0.7457-0.7528)	0.3480(0.2636-0.4324)	0.5020(0.4840- 0.5201)	0.5354(0.4877-0.5841)	0.1829(0.1670-0.1987)
	xgboost	0.6960(0.6630-0.7289)	0.7364(0.7171-0.7558)	0.4727(0.4129-0.5325)	0.5694(0.5303-0.6085)	0.7691(0.7297-0.8069)	0.1567(0.1442-0.1693)
Internal testing	LightGBM	0.7019(0.6475-0.7564)	0.7125(0.6788-0.7462)	0.6856(0.6079-0.7633)	0.6031(0.5544-0.6518)	0.8337(0.7786-0.8873)	0.1834(0.1638-0.2031
	Logistic regression	0.7583(0.7212-0.7953)	0.7250(0.6866-0.7634)	0.8229(0.7916-0.8542)	0.6000(0.5362-0.6638)	0.9047(0.8633-0.9406)	0.1618(0.1567-0.1670)
	Random forest	0.7070(0.6429-0.7710)	0.7167(0.6774-0.7559)	0.6961(0.6056-0.7866)	0.6062(0.5428-0.6697)	0.8439(0.7820-0.8986)	0.1684(0.1532-0.1836)
	svm	0.5130 (0.3227-0.7033)	0.6708 (0.6593-0.6824)	0.6536(0.5500-0.7571)	0.5062(0. 4889-0.5236)	0.6683(0.5897-0.7421)	0.2206(0.2017-0.2394)
	xgboost	0.7307(0.6562-0.8053)	0.7167(0.6774-0.7559)	0.7390(0.6889-0.7890)	0.5906(0.5443-0.6369)	0.8551(0.8048-0.9047)	0.1703(0.1640-0.1765)
External testing	LightGBM	0.8539(0.8004-0.9073)	0.8261(0.7600-0.8962)	0.3667(0.1270-0.6063)	0.5429(0.3604-0.7253)	0.8286(0.7540-0.8962)	0.1051(0.0832-0.1271)
	Logistic regression	0.9394(0.8958-0.9830)	0.8696(0.7842-0.9549)	0.5567(0.3344-0.7789)	0.8833(0.7213-1.0000)	0.9200(0.8678-0.9676)	0.0849(0.0721-0.0976)
	Random forest	0.9074(0.8629-0.9520)	0.8696(0.8034-0.9357)	0.5557(0.2764-0.8350)	0.7476(0.5883-0.9070)	0.8767(0.7852-0.9534)	0.1015(0.0863-0.1168)
	svm	0.8564(0.8083-0.9044)	0.8609(0.7722-0.9496)	0.1791(0.0062-0.3521)	0.5619(0.4085-0.7153)	0.4742(0.2643-0.6807)	0.1120(0.0950-0.1289)
	xgboost	0.8564(0.8057-0.9071)	0.8522(0.8039-0.9005)	0.4033(0.1034-0.7033)	0.5571(0.3908-0.7235)	0.8629(0.7907-0.9189)	0.0966(0.0727-0.1203)

Table 5: Baseline features of the patients in internal and external testing dataset

Variables	Internal testing Mean (±SD), Median (IQR), or n (%)	External testing Mean (±SD), Median (IQR), or n (%)
Sample size	48	23
ΔVAS ≥ 2.6	16 (33.3%)	2 (8.7%)
Age	54.71 (±9.10)	58.10 (±11.98)
Weakness of LL
No	22 (45.8%)	16 (69.6%)
Yes	26 (54.2%)	7 (30.4%)
positive Hoffmann’s sign
No	11 (22.9%)	7 (30.4%)
Yes	37 (77.1%)	16 (69.6%)
*Pre-operative scale*
mJOA score	13.25 (±2.51)	12.13 (±4.38)
VAS of NSP	3.73 (±2.58)	3.09 (±2.33)
VAS of ULP	2.08 (±2.69)	2.00 (±2.68)
SF36-BP	70.50 (±28.42)	68.59 (±23.34)
SF36-PF	68.54 (±26.38)	33.70 (±24.23)
SF36-MH	79.08 (±21.39)	67.65 (±14.91)
SF36-RE	41.67 (±49.82)	53.63 (±38.59)

No competing interests reported.

Development of prediction models and predictors analysis for axial neck pain in patients undergoing cervical laminoplasty based on machine learning

Status:

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

1. Introduction

2. Materials and Methods

2.1 Patient Population

2.2 Baseline Data and Outcomes

2.2.1 Population characteristics: age, gender, smoking history and drinking history.

2.3 Data pre-processing and feature engineering

2.4 Training Machine Learning Models and Performance Evaluation

2.5 Building and validation of a Feature-reduced Model

3. Results

4. Discussion

4.1 Application of ML in spine surgery

4.2 Outcome measure and evaluation of ANP

4.3 Feature importance and risk factors of ANP

4.4 Advantages and limitatons

5. Conclusion

Abbreviations

Declarations

References

Tables

Additional Declarations

Status:

Version 1