Machine Learning Models Based on Hippocampal T2-Weighted-Fluid-Attenuated Inversion Recovery Radiomics for Diagnosis of Posttraumatic Stress Disorder

doi:10.21203/rs.3.rs-2459042/v1

Download PDF

Research Article

Machine Learning Models Based on Hippocampal T2-Weighted-Fluid-Attenuated Inversion Recovery Radiomics for Diagnosis of Posttraumatic Stress Disorder

https://doi.org/10.21203/rs.3.rs-2459042/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

Radiomics is characterized by high-throughput extraction of texture features from medical images for deep mining and analysis to establish meaningful associations between image texture data and specific diseases. Radiomics has demonstrated significant advantages and potential in the diagnosis and evaluation of numerous neurological and psychiatric diseases. However, few studies on its use in the diagnosis of posttraumatic stress disorder (PTSD) have been reported. This study investigated the feasibility of machine learning models based on hippocampal T2-weighted-fluid-attenuated inversion recovery (T2-FLAIR) radiomics for the diagnosis of PTSD.

Methods

We performed a retrospective analysis of the demographic, clinical, and magnetic resonance imaging data of 94 patients with a history of road traffic accident. Regions of interest were manually selected at the bilateral hippocampus on the slices showing the largest respective sizes of the hippocampus. Additionally, the 524 texture features on T2-FLAIR images were extracted. Least absolute shrinkage and selection operator regression was used to screen for the optimal texture features. Thereafter, logistic regression (LR), support vector machine (SVM), and random forest (RF) machine learning models were constructed using the R language for PTSD diagnosis. Receiver operating characteristic curves were used to evaluate the diagnostic performance of each machine learning model.

Results

No statistically significant differences in demographic and clinical characteristics were observed between PTSD and non-PTSD cases after road traffic accident (P > 0.05). However, statistically significant differences in the simplified coping style questionnaire positive/-negative coping scores and PTSD Checklist-Civilian Version scores existed between PTSD and non-PTSD cases at 3 months after road traffic accident (P < 0.01). The performance of three machine learning models in distinguishing PTSD cases from non-PTSD cases was good. In the training and test groups, the area under curves (AUCs) of the LR were 0.829 (95% confidence interval [CI]: 0.717–0.911) and 0.779 (95% CI: 0.584–0.913), with sensitivities and specificities of 74.19% and 77.13%, 76.92% and 80.00%, respectively. The AUCs of the SVM were 0.899 (95% CI: 0.801–0.960) and 0.810 (95% CI: 0.618–0.933), with sensitivities and specificities of 96.77% and 74.29%, 61.54% and 86.67%, respectively. The AUCs of the RF were 0.865 (95% CI: 0.758–0.936) and 0.728 (95% CI: 0.537–0.878), with sensitivities and specificities of 87.10% and 77.14%, 92.31% and 53.33%, respectively.

Conclusions

Machine learning models based on hippocampal T2-FLAIR radiomics have good diagnostic performance for PTSD and can be used as novel neuroimaging biomarkers for the clinical diagnosis of PTSD.

Magnetic resonance imaging

posttraumatic stress disorder

radiomics

hippocampus

machine learning

Posttraumatic stress disorder (PTSD) is a trauma and stressor-related disorder, in which a severe stressful event causes the body to experience a delayed and potentially long-term psychiatric disorder. Core symptoms include intrusive re-experiencing, heightened alertness, avoidance of stimuli associated with traumatic event, and negative psychological alterations in cognition and mood ^{[1, 2]}. A study has shown that road traffic accident is one of the major traumatic events that trigger PTSD, and the prevalence of PTSD among road traffic accident victims is 6–45% ^[3]. The occurrence of PTSD after road traffic accidents not only causes serious physical and psychological dysfunction in patients but also poses a huge burden on public health resources and treatment costs and leads to a loss in productivity ^{[4, 5]}. Therefore, the accurate diagnosis and active intervention of PTSD is of great clinical and social relevance.

The clinical diagnosis and assessment of PTSD mainly relies on self-assessment questionnaires, structured scales, and psychiatric interviews. The use of these tools or methods can be time-consuming and burdensome for clinicians. Additionally, the assessment results are closely related to the clinician's experience and professional training ^{[6, 7]}. Neuroimaging is useful in determining the pathogenesis and pathophysiological changes associated with occurrence and development of PTSD. Functional magnetic resonance imaging (fMRI) reveals activation, functional connectivity, and network abnormalities in many brain regions, such as the anterior cingulate gyrus, thalamus, middle frontal gyrus, parahippocampal gyrus, and amygdala, in patients with PTSD ^{[8, 9]}. Structural MRI shows that PTSD causes a decrease in gray matter and volume reduction in regions, such as the anterior cingulate gyrus, hippocampus, amygdala, and middle temporal gyrus ^[10–12]. Moreover, fMRI and diffusion tensor imaging (DTI) are helpful in the diagnosis of PTSD and in determining the therapeutic efficacy and severity of symptoms ^{[13, 14]}. However, these above MRI techniques are limited by hardware and software, sequence, scan time, and reproducibility and are not widely used for clinical prediction, diagnosis, or assessment of PTSD.

With the key characteristic of high-throughput extraction of texture features from multimodal medical images and data mining to explore disease potential information, radiomics can reveal changes in the tissue, cellular, and genetic levels of diseases. Radiomics has been widely used in the diagnosis and evaluation of numerous neurological and psychiatric diseases, such as Parkinson's disease (PD), Alzheimer's disease (AD), multiple sclerosis, schizophrenia (SCZ), and panic disorder with or without agoraphobia ^[15–21]. Studies have shown that neuroanatomical features and pathophysiological changes are closely associated with diseases, such as AD, PD, and SCZ, and may be potential biomarkers for disease diagnosis. Additionally, these textural features reflect and quantify these anatomical features and microstructural changes ^{[22, 23]}. Moreover, radiomics and machine learning can reveal the neurobiological mechanisms underlying several neurological and psychiatric diseases and have good diagnostic performance ^[22–25]. Our previous study confirmed the changes in texture features in the neostriatum among patients with PD and the good diagnostic performance of radiomics signatures based on conventional T2-weighted (T2W) images for PD ^[20]. Radiomics and machine learning have been demonstrated to have enormous advantages and potential in several pre-clinical and clinical studies on many neurological and psychiatric diseases. However, few studies have focused on their application in PTSD diagnosis.

The hippocampus plays an important role in the stress response process, and it is one of the key limbic systems involved in the occurrence and development of PTSD ^{[26, 27]}. Experimental studies showed apoptosis of hippocampal neurons, glial cell reduction, and microglia activation in PTSD animal models ^[28–30]. MRI showed hippocampal volume reduction, decreased local blood flow, reduced N-acetylaspartate content, and abnormal activity in patients with PTSD ^{[31, 32]}. In our previous study ^[33], we used medial prefrontal cortex (mPFC) T2W image texture analysis to classify PTSD rats and found that changes in mPFC texture feature parameters in PTSD rats may be associated with abnormalities in nuclei, neurons, and glial cells. The texture analysis showed good classification performance. Based on these findings, we hypothesized that hippocampal MRI radiomics combined with machine learning models could be used to diagnose PTSD, although conventional MRI cannot detect visible abnormal signals in the hippocampus. In this study, we analyzed the differences in hippocampal texture feature parameters in the T2-weighted-fluid-attenuated inversion recovery (T2-FLAIR) sequence images of patients with PTSD after road traffic accident. Additionally, we constructed machine learning models based on hippocampal T2-FLAIR radiomics to diagnose PTSD and provide a basis for the clinical diagnosis of PTSD.

Participants

We performed a retrospective analysis of 94 patients with a history of road traffic accidents from July 2020 to November 2022. The sample included 40 males and 54 females with a mean age of 44.79 ± 6.42 years (Range: 34–59 years). The sample size was based on previous similarly relevant studies focused on radiomics or machine learning and a feasibility study conducted during the period of recruitment ^[34–37]. Power analysis revealed that to achieve a power of 0.8 with type I error set at 0.05, assuming an area under ROC of 0.8, a ratio of responders to non-responders of 0.5 was needed ^[38]. For radiomics and machine learning analysis, participants were randomly divided into the training (31 PTSD and 35 non-PTSD cases, 70%) and test (13 PTSD and 15 non-PTSD cases, 30%) groups. Data gathered included clinical data, demographic characteristics, and scale scores. All participants underwent cranial computerized tomography or MRI within 12 hours of the road traffic accident and were followed up for 3 months. Their PTSD statuses were assessed, and the cranial MRI examinations were performed at 3 months after the road traffic accident. PTSD evaluation and diagnosis were made in accordance with the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition ^[2]. The inclusion criteria were: (1) complete clinical and MRI imaging data and images that met the requirements of radiomics analysis; (2) right-handedness; and (3) clear consciousness and good expression and cognitive abilities. The exclusion criteria were: (1) traumatic brain injury (TBI) of all degree; (2) drug and/or alcohol abuse or addiction, use of psychotropic medication; (3) other neurological disorders or hippocampal lesions, such as hippocampal sclerosis, inflammation, tumor and vascular disease involving hippocampus; (4) incomplete clinical and MRI imaging data or poor image quality; (5) history of head injuries or severe traumatic stress events; (6) psychiatric co-morbidities, such as major depressive disorder and other anxiety disorders; and (7) presence of a ferromagnetic material in the body, claustrophobia, or pregnancy.

Survey tools

Scale assessments were performed at the baseline and 3 months after the road traffic accident. Three scales were used in this study: (1) The family, adaptability, partnership, growth, affection, and resolve (APGAR) scale was administered to assess satisfaction with family functioning, including family health, cooperation, growth, affection, and intimacy. High or low scores indicated good or poor family functioning ^[39]. (2) The simplified coping style questionnaire (SCSQ) was used to assess individual coping styles and behaviors ^[40]. The two subscales of the 20-item scale correspond to positive and negative coping situations. Higher scores represented the corresponding coping styles adopted by the participants when they encounter difficulties. (3) The PTSD Checklist-Civilian Version (PCL-C) was used to quantify PTSD symptom severity ^[41]. PCL-C is a 17-item scale of stress symptoms with good internal consistency (Cronbach's α = 0.94), convergent validity (r > 0.75), and test-retest reliability (r = 0.92) ^[42]. PCL-C related to three symptom clusters: re-experiencing, avoidance/numbing, and hyperarousal, with higher scores indicating more severe PTSD symptoms.

MRI image acquisition

A Siemens Verio 3.0T MR scanner (Magnetom Verio, Siemens Medical Solutions, Erlangen, Germany) with an eight-channel head coil was used. The acquisition parameters were as follows: whole-brain axial T1-FLAIR, T2WI, T2-FLAIR, sagittal T1-FLAIR, and hippocampal coronal T2-FLAIR, T1-FLAIR. Whole-brain sagittal T1-FLAIR: time inversion (TI) = 860 ms, time repetition (TR) = 2000 ms, time echo (TE) = 9.9 ms, slice thickness = 5.5 mm, number of excitations (NEX) = 1, matrix size = 320 × 240, field of view (FOV) = 240 × 216 mm; axial T1-FLAIR: TI = 681 ms, TR = 1530 ms, TE = 9 ms, slice thickness = 5.5 mm, NEX = 1, matrix size = 320 × 240, FOV = 240 × 204 mm; axial T2WI: TR = 6000 ms, TE = 96 ms, slice thickness = 5.5 mm, NEX = 1, matrix size = 320 × 320, FOV = 240 × 204 mm; axial T2-FLAIR: TI = 2440 ms, TR = 8500 ms, TE = 94 ms, slice thickness = 5.5 mm, NEX = 1, matrix size = 256 × 179, FOV = 240 × 204 mm. Hippocampal coronal T2-FLAIR: TI = 2440 ms, TR = 8500 ms, TE = 94 ms, slice thickness = 3 mm, NEX = 1, matrix size = 256 × 192, FOV = 240 × 217 mm; T1-FLAIR: TI = 860 ms, TR = 2000 ms, TE = 11 ms, slice thickness = 3 mm, NEX = 2, matrix size = 320 × 240, FOV = 240 × 217 mm.

MRI image analysis and model construction

The MaZda software (v4.6, http://www.eletel.p.lodz.pl/programy/mazda) was used to extract hippocampal texture features. The three coronal T2-FLAIR slices showing the greatest possible view of bilateral hippocampus were identified by two radiologists (SZ and YS had 11 and 9 years of experience, respectively). They were then imported into MaZda with reference to other sequence images and normalized for image grayscale (µ ± 3SD [µ: gray-level mean, SD: gray-level standard deviation]). As described by previous studies ^{[43, 44]}, two radiologists (SZ and YS) performed manual selection of bilateral hippocampal region of interest (ROI) (ROI included the hippocampusas much as possible) in each of the 40 cases, using the other sequence images as anatomical reference (Fig. 1). SZ repeated the selection once and completed selection of the remaining 54 cases. The inter- and intra-group agreement between the two radiologists' ROI selections was assessed. This segmentation process resulted in six ROIs per case. To account for sampling variability, we averaged texture features over slice without losing laterality information, leading to 524 texture features (262 per hippocampus) in each case. In this study, the texture feature classes extracted by MaZda included gray-level histogram, gray-level co-occurrence matrix, absolute gradient matrix, run-length matrix, and autoregressive model. The optimal features were screened using least absolute shrinkage and selection operator (LASSO) regression with 10-fold cross-validation in the R language’s "glmnet" package. The screened optimal features underwent linear fusion to calculate the radiomics score (Rad-score) in each case. Thereafter, logistic regression (LR), support vector machine (SVM), and random forest (RF) machine learning models were constructed using the R language’s "glm", "kernlab", and "randomForest" packages, respectively.

Statistical analysis

SPSS 22.0 software (v22.0; SPSS Inc., Chicago, IL, USA) was used for statistical analysis. Quantitative data were expressed as mean ± SD or M (Qn). The F test was used for variance homogeneity. The Chi-square, Student’s t/t′, and Mann–Whitney U tests were used to compare the differences in demographic and clinical characteristics, scale scores, and RAD-scores between the groups. Inter- and intra-observer consistency of two radiologists’ manual selection of the hippocampal ROIs was evaluated using inter- and intra-class correlation coefficients (ICCs). The R software (v3.4.3; R Foundation for Statistical Computing, Vienna, Austria) was used to screen classification of optimal features and construct machine learning models. Spearman correlation was used to analyze the correlation between RAD-scores and PCL-C total scores. Receiver operating characteristic (ROC) curves were used to evaluate the diagnostic performance of each machine learning model, and the MedCalc software (v20.019; MedCalc Software Ltd, Ostend, Belgium; https://www.medcalc.org; 2021) Delong test was used to compare the differences in the area under the curve (AUC) among the models. The goodness-of-fit of the machine learning models was evaluated using the Hosmer–Lemeshow test. A two-tailed P < 0.05 was considered statistically significant in all statistical analyses.

Demographics and clinical characteristics of participants

In the training and test groups, no statistically significant differences in age, gender, body mass index, monthly per capita income, physical exertion, health insurance, education level, or APGAR scores were observed between PTSD and non-PTSD cases (P > 0.05). Additionally, no statistically significant differences in SCSQ-positive/-negative coping scores, PCL-C scores for each dimension (re-experiencing, avoidance/numbing, and hyperarousal), and PCL-C total scores were observed between PTSD and non-PTSD cases at baseline (P > 0.05). The differences in SCSQ-positive/negative coping scores, PCL-C scores for each dimension, and PCL-C total scores between patients with PTSD and those without PTSD at 3 months after the road traffic accident were statistically significant (P < 0.01) (Tables 1 and 2).

Table 1

Demographic and clinical characteristics of participants
	Training group (n = 66)				Test group (n = 28)
	PTSD	non-PTSD	χ²/t/t′	P	PTSD	non-PTSD	χ²/t/t′	P
Age(years)	42.97 ± 5.81	44.54 ± 6.39	-1.043	0.301	44.76 ± 6.02	46.47 ± 6.77	-0.696	0.492
Gender(male/female)	12/19	15/20	0.117	0.732	7/6	6/9	0.537	0.464
Body Mass Index(kg/m²)	21.82 ± 3.11	21.99 ± 3.03	-0.221	0.826	22.26 ± 3.42	23.17 ± 3.66	-0.671	0.508
Monthly per capita income (yuan)	2508.86 ± 644.54	2589.58 ± 795.75	-0.449	0.655	2773.74 ± 959.10	2609.58 ± 722.12	0.516	0.610
Physical work [N (%)]	17(54.84%)	16(45.71%)	0.547	0.459	6(46.15%)	8(53.33%)	0.144	0.705
Health insurance [N (%)]	23(74.19%)	25(71.43%)	0.063	0.801	9(69.23%)	9(60.00%)	0.258	0.611
Education years	9.94 ± 2.72	10.37 ± 2.94	-0.622	0.536	8.85 ± 3.24	10.60 ± 3.58	-1.351	0.188

Table 2

The scales scores of participants
		Training group (n = 66)				Test group (n = 28)
		PTSD	non-PTSD	t/t′	P	PTSD	non-PTSD	t/t′	P
Baseline
APGAR questionnaire		6.87 ± 0.92	7.03 ± 1.01	-0.657	0.513	6.92 ± 1.26	7.13 ± 0.92	-0.511	0.614
SCSQ	Positive copying	19.58 ± 4.18	20.09 ± 3.03	-0.556	0.581	20.08 ± 3.31	20.40 ± 3.14	-0.265	0.793
SCSQ	Negative copying	11.06 ± 2.41	10.97 ± 2.49	0.154	0.878	10.77 ± 1.54	11.13 ± 2.23	-0.495	0.625
PCL-C	Re-experiencing	8.84 ± 1.35	8.54 ± 1.36	0.887	0.377	8.31 ± 1.03	8.07 ± 1.28	0.543	0.592
	Avoidance	9.81 ± 1.38	9.92 ± 1.40	-0.315	0.754	9.77 ± 1.48	9.27 ± 1.16	1.005	0.324
	Hyperarousal	7.03 ± 1.14	6.94 ± 1.13	0.319	0.751	7.62 ± 1.26	7.73 ± 1.22	-0.251	0.804
	Total	25.68 ± 2.47	25.40 ± 1.90	0.507	0.614	25.69 ± 2.06	25.07 ± 2.19	0.776	0.445
3 months after road traffic accident
APGAR questionnaire		6.97 ± 1.54	7.17 ± 1.34	-0.575	0.567	7.31 ± 0.95	7.20 ± 0.78	0.331	0.743
SCSQ	Positive copying	17.48 ± 3.31	21.47 ± 2.33	-5.723	0.000	17.62 ± 2.60	20.93 ± 1.87	-3.916	0.001
SCSQ	Negative copying	13.06 ± 1.89	10.49 ± 1.42	6.294	0.000	13.75 ± 2.21	10.87 ± 1.06	4.333	0.000
PCL-C	Re-experiencing	13.90 ± 2.68	8.57 ± 1.50	9.813	0.000	12.77 ± 2.71	8.26 ± 1.58	5.458	0.000
	Avoidance	19.45 ± 1.95	9.77 ± 2.03	19.707	0.000	18.46 ± 3.36	9.13 ± 1.69	9.490	0.000
	Hyperarousal	10.39 ± 2.17	6.49 ± 0.95	9.252	0.000	11.23 ± 2.35	7.13 ± 1.14	5.740	0.000
	Total	43.74 ± 4.41	24.83 ± 3.19	20.129	0.000	42.46 ± 6.31	24.53 ± 2.30	9.708	0.000

LASSO regression screening the optimal texture features

The intra- and inter-group ICCs for the manually selected hippocampal ROIs of the two radiologists were 0.886 and 0.843, respectively, indicating high consistency. The penalty term lambda was determined based on the 10-fold cross-validation LASSO regression. To determine the minimization criteria of binomial deviance, the optimized lambda value for the left dotted vertical line was selected [the lambda value of 0.032 with log (lambda) = − 3.442] (Fig. 2). After dimensionality reduction via LASSO regression, 11 optimal features with non-zero coefficients remained in the hippocampal T2-FLAIR radiomics signature. The 11 optimal features can be classified into three major categories: the histogram, co-occurrence matrix, and autoregressive model, respectively. Afterwards, the optimal texture features were used to calculate the Rad-score: Rad-score = 29.680 − 0.156 × R_MinNorm − 8.943 × R_S(0,1)InvDfMom − 0.001 × R_S(1,1)SumOfSqs − 0.001 × R_S(0,2)SumVarnc − 0.002 × R_S(4,4)DifVarnc + 10.155 × R_S(5,0)InvDfMom − 0.045 × R_Teta1 − 2.253 × L_S(0,3)DifEntrp − 9.223 × L_S(3,3)InvDfMom + 1.009 × L_S(5,-5)SumEntrp − 0.323 × L_Teta3.

Correlation between RAD-scores and PCL-C scores

In the training and test groups, the RAD-scores of 2.366 (0.544) and 2.234 (0.372) for those with PTSD were greater than the RAD-scores of 1.883 (0.564) and 1.784 (0.526) for those without PTSD. The difference between the PTSD and non-PTSD cases was statistically significant (Z_{training group} = − 4.169, P_{training group} = 0.000; Z_{test group} = − 2.096, P_{test group} = 0.037). To analyze the correlation between hippocampal T2-FLAIR texture features and severity of PTSD symptoms, we chose RAD-scores and PCL-C total scores to construct the scatter plots. The RAD-scores were positively correlated with the PCL-C total scores in the training and test groups (Fig. 3).

Diagnostic performance of machine learning models based on hippocampal T2-FLAIR radiomics

The optimal texture features screened by LASSO regression were used to construct LR, SVM, and RF machine learning models. The performance of each model for diagnosing PTSD is shown in Table 3 and Fig. 4. In the training group, the highest AUC of SVM was 0.899, the differences in AUCs among the three machine learning models were not statistically significant (Z_{RF vs SVM} = 0.552, P_{RF vs SVM} = 0.581; Z_{LR vs SVM} = 1.128, P_{LR vs SVM} = 0.259; Z_{RF vs LR} = 0.646, P_{RF vs LR} = 0.517). In the test group, RF had the lowest AUC of 0.728, the differences in AUCs among the three machine learning models were not statistically significant (Z_{RF vs SVM} = 0.640, P_{RF vs SVM} = 0.522; Z_{RF vs LR} = 0.379, P_{RF vs LR} = 0.705; Z_{LR vs SVM} = 0.259, P_{LR vs SVM} = 0.796). The Hosmer–Lemeshow test showed that the LR, SVM, and RF machine learning models had acceptable goodness-of-fit in the training and test groups (P_{LR−training} = 0.204, P_{SVM−training} = 0.106, P_{RF−training} = 0.546; P_LR−test = 0.381, P_SVM−test = 0.586, P_RF−test = 0.745; respectively).

Table 3

Diagnostic performance of machine learning models
Machine learning models	Training group			Test group
Machine learning models	AUC (95%CI)	Sensitivity	Specificity	AUC (95%CI)	Sensitivity	Specificity
LR	0.829 (0.717–0.911)	74.19%	77.13%	0.779 (0.584–0.913)	76.92%	80.00%
SVM	0.899 (0.801–0.960)	96.77%	74.29%	0.810 (0.618–0.933)	61.54%	86.67%
RF	0.865 (0.758–0.936)	87.10%	77.14%	0.728 (0.537–0.878)	92.31%	53.33%

In recent years, several studies on the prediction and diagnosis of PTSD using machine learning methods (collect the symptoms, demographics, trauma types, psychometric information, and MRI data of patients who experienced traumatic events, followed with text mining or machine learning to predict and diagnose PTSD) have been conducted. These methods mostly had AUCs of 0.75–0.96, ranging in accuracy from 57.58% to 92.5% ^[45-48]. MRI techniques, such as fMRI and DTI, combined with machine learning have been used in the prediction, diagnosis, and assessment of PTSD ^[49,50]. However, machine learning models based on conventional MRI radiomics for PTSD diagnosis have not been reported.

Previously, we used stepwise discriminant analysis (SDA) and LASSO regression methods to screen mPFC texture features based on T2W images and build radiomics classification models for rats in the control and PTSD groups ^[33]. SDA analysis showed an overall classification accuracy of 86.5% (non-cross-validation) and 80.4% (cross-validation). Radiomics signatures based on LASSO regression showed that the AUCs for classifying PTSD rats between the control group and each PTSD group were greater than 0.935, and the AUCs among four PTSD groups were greater than 0.889. All these results indicated that mPFC T2W texture analysis possessed good classification performance. The hippocampus plays an important role in the occurrence and development of PTSD. Clinical studies ^[31,51] have shown that the hippocampal volume shrinkage and decreased N-acetylaspartate levels in patients with PTSD are closely related to pathological changes, such as hippocampal neuronal loss and glial cell reduction, which have been verified by experimental studies ^[30,52]. Based on our previous study and relevant theories, the present study used LR, SVM, and RF machine learning models based on hippocampal T2-FLAIR radiomics to diagnose PTSD after road traffic accidents. The results showed that, in the training group, the AUCs of LR, SVM, and RF for diagnosing PTSD were 0.829, 0.899, and 0.865, with sensitivities of 74.19%, 96.77%, and 87.10%, specificities of 77.13%, 74.29%, and 77.14%, respectively. The AUCs of the three machine learning models in the test group were 0.779, 0.810, and 0.728, with sensitivities of 76.92%, 61.54%, and 92.31%, specificities of 80.00%, 86.67%, and 53.33%, respectively. These results demonstrate that the machine learning models based on hippocampal T2-FLAIR radiomics have good diagnostic performance. However, compared to our previous experimental study, this study had lower AUCs for the diagnosis of PTSD. With the exception of differences in the selection of brain region, the AUCs were lower probably due to the following reasons. First, few confounding factors and biases were generated in the experimental study. Second, the PTSD animal model highly simulated the pathogenesis and pathological changes of PTSD in humans. However, significant differences in pathophysiology and mental, emotional, and social environments persist. Third, the differences in age, gender, physical exertion, and educational level between PTSD and non-PTSD cases included in this study were not statistically significant. Nevertheless, for each individual, many of these characteristics or factors and their interactions will be important in the occurrence and development of PTSD ^[53].

Several clinical studies in recent years have demonstrated that machine learning combined with fMRI can predict and assess the status, severity, and symptom clusters in patients with PTSD. Furthermore, it can determine comorbidity with major depression and reveal variables associated with the onset and classification of PTSD in trauma survivors ^[54-56]. Liu et al. ^[57] used the combination of multi-level features extracted from fMRI images and multi-kernel learning to classify PTSD cases and healthy controls with an accuracy of 92.5%. Saba et al. ^[49] used machine learning combined with rs-fMRI to classify PTSD cases and healthy controls with an accuracy of 93.7–99.2% (K-nearest neighbor and SVM with radial basis function kernel) in the training, validation, and test groups. This study had a lower diagnostic performance than previous clinical studies due to the following reasons: differences in the types of trauma, assessment methods, demographic characteristics of participants, MRI techniques, brain region selection, or machine learning algorithms. All of these factors led to differences in the classification and diagnostic performance. However, compared to the above-mentioned fMRI, DTI, and other imaging techniques, conventional MRI has many advantages, including a shorter examination duration, greater convenience, and no hardware or software limitations, making it convenient for use in daily clinical settings.

In this study, although we used 10-fold cross-validation LASSO regression to reduce the effect of model over-fitting (LASSO regression is a sparse learning method for high-dimensional data and commonly used for feature optimization and risk factor screening ^[20,58,59]) and the Hosmer–Lemeshow test to assess the goodness-of-fit of each model, the small sample size was an important factor for over-fitting or insufficient expression in the machine learning models. This study preliminarily showed that machine learning models based on hippocampal T2-FLAIR radiomics are promising as a potential imaging method for PTSD diagnosis after road traffic accidents. However, further multicenter studies with large samples are needed.

This study had some limitations. First, it was a retrospective study, had a small sample size and short follow-up period, and excluded individuals with any complication of TBI. Second, ROIs were manually selected from the largest area of hippocampal T2-FLAIR images. The hippocampal substructures, other brain regions, and other sequence images were not analyzed in detail. Third, only LR, SVM, and RF machine learning algorithms were used; other algorithms were not investigated.

The preliminary findings of this study suggest that the machine learning models based on hippocampal T2-FLAIR radiomics achieves good diagnostic performance for PTSD after road traffic accidents. These models are expected to be used as novel neuroimaging biomarkers for the clinical diagnosis and assessment of PTSD. Follow-up multicenter prospective studies with large sample sizes are needed to further optimize and validate the machine learning models based on multi-brain region and multi-sequence MRI radiomics, and explore the stability and clinical application of the models by including patients with PTSD and/or TBI.

APGAR: Adaptability, partnership, growth, affection and resolve

AUC: Area under the curve

DTI: Diffusion tensor imaging

fMRI: Functional magnetic resonance imaging

ICC: Intraclass correlation coefficient

LASSO: Least absolute shrinkage and selection operator

LR: Logistic regression

mPFC: Medial prefrontal cortex

PCL-C: PTSD checklist-civilian version

PTSD: Posttraumatic stress disorder

Rad-score: Radiomics score

RF: Random forest

ROC: Receiver operating characteristic

ROI: Region of interest

rs-fMRI: Resting state-functional magnetic resonance imaging

SCSQ: Simplified coping style questionnaire

SDA: Step discriminant analysis

SVM: Support vector machine

TBI: Traumatic brain injury

T2-FLAIR: T2-weighted-fluid-attenuated inversion recovery

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Acknowledgements

We would like to thank Prof. Yuxiu Shi for her assistance. We would also like to express our gratitude to the reviewers for their precious time and valuable comments, as well as the patients who completed our investigation in the present study.

Funding

This research was supported by the science and technology fund from Department of Education of Liaoning Province (No. JYTQN2020013), doctoral research startup fund from Department of Science and Technology of Liaoning Province (No. 2019-BS-099).

Author information

Authors and Affiliations

Department of Radiology, First Affiliated Hospital of Jinzhou Medical University, Jinzhou, Liaoning, China

Shilei Zheng, Yu Sun & Xianglin Zhang

Department of Radiology, Dongping People’s Hospital, Dongping, Shandong, China

Xuekai Zhao

Medical Imaging Center, Taian Central Hospital, Taian, Shandong, China

Han Wang

Department of Neurology, First Affiliated Hospital of Jinzhou Medical University, Jinzhou, Liaoning, China

Fan Zhang & Li-e Zang

Department of Emergency, First Affiliated Hospital of Jinzhou Medical University, Jinzhou, Liaoning, China

Jufeng Sun

Department of Stomatology, First Affiliated Hospital of Jinzhou Medical University, Jinzhou, Liaoning, China

Lili Zhang

Authors’ contribution

SZ designed the study with LZ and HW. SZ, XZ, YS, JS, XZ, LZ and FZ performed data collection, MRI examination and analysis. The first draft of the manuscript was written by SZ, HW and LZ, and all authors commented on previous versions of the manuscript. All authors had reviewed this manuscript and approved its submission.

Corresponding author

Correspondence to Lili Zhang

Ethics declarations

Ethics approval and consent to participate

The study was conducted in accordance with the Institutional Research Ethics guidelines and ethical guidelines involving human participation (i.e., Helsinki Declaration). Formal ethics approval was granted by Ethical Review Committee, First Affiliated Hospital of Jinzhou Medical University (Ref. No.: KYLL–202026). Informed consent was obtained from all participants. All methods were performed in accordance with relevant ethical guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no potential conflict of interest in the publication of this research output.

Islam MS, Ferdous MZ, Sujan MSH, Tasnim R, Masud JHB, Kundu S, et al. The psychometric properties of the bangla posttraumatic stress disorder checklist for DSM-5 (PCL-5): preliminary reports from a large-scale validation study. BMC Psychiatry. 2022; 22(1): 280. DOI: 10.1186/s12888-022-03920-4.
American Psychiatric Association (APA). Diagnostic and statistical manual of mental disorders, 5th ed. Arlington VA: American Psychiatric Publishing. 2013.
Heron-Delaney M, Kenardy J, Charlton E, Matsuoka Y. A systematic review of predictors of posttraumatic stress disorder (PTSD) for adult road traffic crash survivors. Injury. 2013; 44(11): 1413–22. DOI: 10.1016/j.injury.2013.07.011.
Ziobrowski HN, Kennedy CJ, Ustun B, House SL, Beaudoin FL, An X, et al. Development and validation of a model to predict posttraumatic stress disorder and major depression after a motor vehicle collision. JAMA Psychiatry. 2021; 78(11): 1228–37. DOI: 10.1001/JAMAPSYCHIATRY.2021.2427.
Arora D, Belsiyal CX, Rawat VS. Prevalence and determinants of posttraumatic stress disorder and depression among survivors of motor vehicle accidents from a hilly Indian state. Indian J Psychiatry. 2021; 63(3): 250–7. DOI: 10.4103/psychiatry.IndianJPsychiatry_1059_20.
Carvajal C. Posttraumatic stress disorder as a diagnostic entity-clinical perspectives. Dialogues Clin Neurosci. 2018; 20(3): 161–8. DOI: 10.31887/DCNS.2018.20.3/ccarvajal.
Jorge RE. Posttraumatic stress disorder. Continuum (Minneap Minn). 2015; 21(3 Behavioral Neurology and Neuropsychiatry): 789–805. DOI: 10.1212/01.CON.0000466667.20403.b1
Sydnor VJ, Cieslak M, Duprat R, Deluisi J, Flounders MW, Long H, et al. Cortical-subcortical structural connections support transcranial magnetic stimulation engagement of the amygdala. Sci Adv. 2022; 8(25): eabn5803. DOI: 10.1126/sciadv.abn5803.
Bao W, Gao Y, Cao L, Li H, Liu J, Liang K, et al. Alterations in large-scale functional networks in adult posttraumatic stress disorder: a systematic review and meta-analysis of resting-state functional connectivity studies. Neurosci Biobehav Rev. 2021; 131: 1027–36. DOI: 10.1016/j.neubiorev.2021.10.017.
Li L, Wu M, Liao Y, Ouyang L, Du M, Lei D, et al. Grey matter reduction associated with posttraumatic stress disorder and traumatic stress. Neurosci Biobehav Rev. 2014; 43: 163–72. DOI: 10.1016/j.neubiorev.2014.04.003.
O'Doherty DC, Chitty KM, Saddiqui S, Bennett MR, Lagopoulos J. A systematic review and meta-analysis of magnetic resonance imaging measurement of structural volumes in posttraumatic stress disorder. Psychiatry Res. 2015; 232(1): 1–33. DOI: 10.1016/j.pscychresns.2015.01.002.
Bromis K, Calem M, Reinders AATS, Williams SCR, Kempton MJ. Meta-analysis of 89 structural MRI studies in posttraumatic stress disorder and comparison with major depressive disorder. Am J Psychiatry. 2018; 175(10): 989–98. DOI: 10.1176/appi.ajp.2018.17111199.
Zhutovsky P, Thomas RM, Olff M, van Rooij SJH, Kennis M, van Wingen GA, et al. Individual prediction of psychotherapy outcome in posttraumatic stress disorder using neuroimaging data. Transl Psychiatry. 2019; 9(1): 326. DOI: 10.1038/s41398-019-0663-7.
Kennis PhD M, van Rooij PhD SJH, Reijnen MSc A, Geuze PhD E. The predictive value of dorsal cingulate activity and fractional anisotropy on long-term PTSD symptom severity. Depress Anxiety. 2017; 34(5): 410–8. DOI: 10.1002/da.22605.
Bang M, Eom J, An C, Kim S, Park YW, Ahn SS, et al. An interpretable multiparametric radiomics model for the diagnosis of schizophrenia using magnetic resonance imaging of the corpus callosum. Transl Psychiatry. 2021; 11(1): 462. DOI: 10.1038/s41398-021-01586-2.
Guan XJ, Guo T, Zhou C, Gao T, Wu JJ, Han V, et al. A multiple-tissue-specific magnetic resonance imaging model for diagnosing Parkinson's disease: a brain radiomics study. Neural Regen Res. 2022; 17(12): 2743–9. DOI: 10.4103/1673-5374.339493.
Tang L, Wu X, Liu H, Wu F, Song R, Zhang W, et al. Individualized prediction of early Alzheimer's disease based on magnetic resonance imaging radiomics, clinical, and laboratory examinations: a 60-month follow-up study. J Magn Reson Imaging. 2021; 54(5): 1647–57. DOI: 10.1002/jmri.27689.
Cui LB, Xu X, Cao F. Building the precision medicine for mental disorders via radiomics/machine learning and neuroimaging. Front Neurosci. 2021; 15: 685005. DOI: 10.3389/fnins.2021.685005.
Luo X, Piao S, Li H, Li Y, Xia W, Bao Y, et al. Multi-lesion radiomics model for discrimination of relapsing-remitting multiple sclerosis and neuropsychiatric systemic lupus erythematosus. Eur Radiol. 2022; 32(8): 5700–10. DOI: 10.1007/s00330-022-08653-2.
Liu P, Wang H, Zheng S, Zhang F, Zhang X. Parkinson’s disease diagnosis using neostriatum radiomic features based on T2-weighted magnetic resonance imaging. Front Neurol. 2020; 11: 248. DOI: 10.3389/fneur.2020.00248.
Bang M, Park YW, Eom J, Ahn SS, Kim J, Lee SK, et al. An interpretable radiomics model for the diagnosis of panic disorder with or without agoraphobia using magnetic resonance imaging. J Affect Disord. 2022; 305: 47–54. DOI: 10.1016/j.jad.2022.02.072
Corrias G, Micheletti G, Barberini L, Suri JS, Saba L. Texture analysis imaging "what a clinical radiologist needs to know". Eur J Radiol. 2022; 146: 110055. DOI: 10.1016/j.ejrad.2021.110055.
Cai JH, He Y, Zhong XL, Lei H, Wang F, Luo GH, et al. Magnetic resonance texture analysis in Alzheimer's disease. Acad Radiol. 2020; 27(12): 1774–83. DOI: 10.1016/j.acra.2020.01.006.
Lanka P, Rangaprakash D, Dretsch MN, Katz JS, Denney TS Jr, Deshpande G. Supervised machine learning for diagnostic classification from large-scale neuroimaging datasets. Brain Imaging Behav. 2020; 14(6): 2378–416. DOI: 10.1007/s11682-019-00191-8.
Korda AI, Andreou C, Rogg HV, Avram M, Ruef A, Davatzikos C, et al. Identification of texture MRI brain abnormalities on first-episode psychosis and clinical high-risk subjects using explainable artificial intelligence. Transl Psychiatry. 2022; 12(1): 481. DOI: 10.1038/s41398-022-02242-z.
Joshi SA, Duval ER, Kubat B, Liberzon I. A review of hippocampal activation in post-traumatic stress disorder. Psychophysiology. 2020; 57(1): e13357. DOI: 10.1111/psyp.13357.
Harnett NG, Goodman AM, Knight DC. PTSD-related neuroimaging abnormalities in brain function, structure, and biochemistry. Exp Neurol. 2020; 330: 113331. DOI: 10.1016/j.expneurol.2020.113331.
Li S, Liao Y, Dong Y, Li X, Li J, Cheng Y, et al. Microglial deletion and inhibition alleviate behavior of post-traumatic stress disorder in mice. J Neuroinflammation. 2021; 18(1): 7. DOI: 10.1186/s12974-020-02069-9.
Han F, Xiao B, Wen L, Shi Y. Effects of fluoxetine on the amygdala and the hippocampus after administration of a single prolonged stress to male wistar rates: in vivo proton magnetic resonance spectroscopy findings. Psychiatry Res. 2015; 232(2): 154–61. DOI: 10.1016/j.pscychresns.2015.02.011.
Han F, Xiao B, Wen L. Loss of glial cells of the hippocampus in a rat model of post-traumatic stress disorder. Neurochem Res. 2015; 40(5): 942–51. DOI: 10.1007/s11064-015-1549-6.
Kamiya K, Abe O. Imaging of posttraumatic stress disorder. Neuroimaging Clin N Am. 2020; 30(1): 115–23. DOI: 10.1016/j.nic.2019.09.010.
Del Casale A, Ferracuti S, Barbetti AS, Bargagna P, Zega P, Iannuccelli A, et al. Grey matter volume reductions of the left hippocampus and amygdala in PTSD: a coordinate-based meta-analysis of magnetic resonance imaging studies. Neuropsychobiology. 2022; 81(4): 257–64. DOI: 10.1159/000522003.
Zheng S, Wang H, Han F, Chu J, Zhang F, Zhang X, et al. Detection of microstructural medial prefrontal cortex changes using magnetic resonance imaging texture analysis in a post-traumatic stress disorder rat model. Front Psychiatry. 2022; 13: 805851. DOI: 10.3389/fpsyt.2022.805851.
Du Y, Zhang S, Fang Y, Qiu Q, Zhao L, Wei W, et al. Radiomic features of the hippocampus for diagnosing early-onset and late-onset Alzheimer's disease. Front Aging Neurosci. 2022; 13: 789099. DOI: 10.3389/fnagi.2021.789099.
Zhang J, Richardson JD, Dunkley BT. Classifying post-traumatic stress disorder using the magnetoencephalographic connectome and machine learning. Sci Rep. 2020; 10(1): 5937. DOI: 10.1038/s41598-020-62713-5.
Chu H, Lin X, He J, Pang P, Fan B, Lei P, et al. Value of MRI radiomics based on enhanced T1WI images in prediction of meningiomas grade. Acad Radiol. 2021; 28(5): 687–93. DOI: 10.1016/j.acra.2020.03.034.
Gong J, Cui LB, Xi YB, Zhao YS, Yang XJ, Xu ZL, et al. Predicting response to electroconvulsive therapy combined with antipsychotics in schizophrenia using multi-parametric magnetic resonance imaging. Schizophr Res. 2020; 216: 262–71. DOI: 10.1016/j.schres.2019.11.046.
Obuchowski NA, Lieber ML, Wians FH Jr. ROC curves in clinical chemistry: uses, misuses, and possible solutions. Clin Chem. 2004; 50(7): 1118–25. DOI: 10.1373/clinchem.2004.031823.
Hou T, Yin Q, Cai W, Song X, Deng W, Zhang J, et al. Posttraumatic stress symptoms among health care workers during the COVID-19 epidemic: the roles of negative coping and fatigue. Psychol Health Med. 2022; 27(2): 367–78. DOI: 10.1080/13548506.2021.1921228.
Ayer L, Ramchand R, Karimi G, Wong EC. Co-occurring alcohol and mental health problems in the military: prevalence, disparities, and service utilization. Psychol Addict Behav. 2022; 36(4): 419–27. DOI: 10.1037/adb0000804.
Karstoft KI, Andersen SB, Bertelsen M, Madsen T. Diagnostic accuracy of the posttraumatic stress disorder checklist-civilian version in a representative military sample. Psychol Assess. 2014; 26(1): 321–5. DOI: 10.1037/a0034889.
Ruggiero KJ, Del Ben K, Scotti JR, Rabalais AE. Psychometric properties of the PTSD checklist-civilian version. J Trauma Stress. 2003; 16(5): 495–502. DOI: 10.1023/A:1025714729117.
Ranjbar S, Velgos SN, Dueck AC, Geda YE, Mitchell JR; Alzheimer's disease neuroimaging initiative. Brain MR radiomics to differentiate cognitive disorders. J Neuropsychiatry Clin Neurosci. 2019; 31(3): 210–9. DOI: 10.1176/appi.neuropsych.17120366.
Jafari-Khouzani K, Elisevich K, Patel S, Smith B, Soltanian-Zadeh H. FLAIR signal and texture analysis for lateralizing mesial temporal lobe epilepsy. Neuroimage. 2010; 49(2): 1559–71. DOI: 10.1016/j.neuroimage.2009.08.064.
Karstoft KI, Galatzer-Levy IR, Statnikov A, Li Z, Shalev AY; members of Jerusalem Trauma Outreach and Prevention Study (J-TOPS) group. Bridging a translational gap: using machine learning to improve the prediction of PTSD. BMC Psychiatry. 2015; 15: 30. DOI: 10.1186/s12888-015-0399-8.
Schultebraucks K, Sijbrandij M, Galatzer-Levy I, Mouthaan J, Olff M, van Zuiden M. Forecasting individual risk for long-term posttraumatic stress disorder in emergency medical settings using biomedical data: a machine learning multicenter cohort study. Neurobiol Stress. 2021; 14: 100297. DOI: 10.1016/j.ynstr.2021.100297.
Ramos-Lima LF, Waikamp V, Antonelli-Salgado T, Passos IC, Freitas LHM. The use of machine learning techniques in trauma-related disorders: a systematic review. J Psychiatr Res. 2020; 121: 159–72. DOI: 10.1016/j.jpsychires.2019.12.001.
Schultebraucks K, Galatzer-Levy IR. Machine learning for prediction of posttraumatic stress and resilience following trauma: an overview of basic concepts and recent advances. J Trauma Stress. 2019; 32(2): 215–25. DOI: 10.1002/jts.22384.
Saba T, Rehman A, Shahzad MN, Latif R, Bahaj SA, Alyami J. Machine learning for post-traumatic stress disorder identification utilizing resting-state functional magnetic resonance imaging. Microsc Res Tech. 2022; 85(6): 2083–94. DOI: 10.1002/jemt.24065.
Zhu X, Suarez-Jimenez B, Zilcha-Mano S, Lazarov A, Arnon S, Lowell AL, et al. Neural changes following equine-assisted therapy for posttraumatic stress disorder: a longitudinal multimodal imaging study. Hum Brain Mapp. 2021; 42(6): 1930–9. DOI: 10.1002/hbm.25360.
Kunimatsu A, Yasaka K, Akai H, Kunimatsu N, Abe O. MRI findings in posttraumatic stress disorder. J Magn Reson Imaging. 2020; 52(2): 380–96. DOI: 10.1002/jmri.26929.
Araki M, Fuchikami M, Omura J, Miyagi T, Nagashima N, Okamoto Y, et al. The role of glucocorticoid receptors in the induction and prevention of hippocampal abnormalities in an animal model of posttraumatic stress disorder. Psychopharmacology (Berl). 2020; 237(7): 2125–37. DOI: 10.1007/s00213-020-05523-x.
Al Jowf GI, Ahmed ZT, An N, Reijnders RA, Ambrosino E, Rutten BPF, et al. A public health perspective of post-traumatic stress disorder. Int J Environ Res Public Health. 2022; 19(11): 6474. DOI: 10.3390/ijerph19116474.
Zilcha-Mano S, Zhu X, Suarez-Jimenez B, Pickover A, Tal S, Such S, et al. Diagnostic and predictive neuroimaging biomarkers for posttraumatic stress disorder. Biol Psychiatry Cogn Neurosci Neuroimaging. 2020; 5(7): 688–96. DOI: 10.1016/j.bpsc.2020.03.010.
Ben-Zion Z, Zeevi Y, Keynan NJ, Admon R, Kozlovski T, Sharon H, et al. Multi-domain potential biomarkers for post-traumatic stress disorder (PTSD) severity in recent trauma survivors. Transl Psychiatry. 2020; 10(1): 208. DOI: 10.1038/s41398-020-00898-z.
Sheynin S, Wolf L, Ben-Zion Z, Sheynin J, Reznik S, Keynan JN, et al. Deep learning model of fMRI connectivity predicts PTSD symptom trajectories in recent trauma survivors. Neuroimage. 2021; 238: 118242. DOI: 10.1016/j.neuroimage.2021.118242.
Liu F, Xie B, Wang Y, Guo W, Fouche JP, Long Z, et al. Characterization of post-traumatic stress disorder using resting-state fMRI with a multi-level parametric classification approach. Brain Topogr. 2015; 28(2): 221–37. DOI: 10.1007/s10548-014-0386-2.
Qiao J, Sui R, Zhang L, Wang J. Construction of a risk model associated with prognosis of post-stroke depression based on magnetic resonance spectroscopy. Neuropsychiatr Dis Treat. 2020; 16: 1171–80. DOI: 10.2147/NDT.S245129.
Yuan X, Zhang L, Sui R, Wang Z. A risk prediction model of post-stroke cognitive impairment based on magnetic resonance spectroscopy imaging. Neurol Res. 2021; 43(8): 642–52. DOI: 10.1080/01616412.2021.1908659.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Machine Learning Models Based on Hippocampal T2-Weighted-Fluid-Attenuated Inversion Recovery Radiomics for Diagnosis of Posttraumatic Stress Disorder

Status:

Version 1

Abstract

Figures

Introduction

Materials And Methods

Participants

Survey tools

MRI image acquisition

MRI image analysis and model construction

Statistical analysis

Results

Demographics and clinical characteristics of participants

LASSO regression screening the optimal texture features

Correlation between RAD-scores and PCL-C scores

Diagnostic performance of machine learning models based on hippocampal T2-FLAIR radiomics

Discussion

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1