Predicting Suicide Outcomes: An Analysis of Key Factors and Machine Learning Models

doi:10.21203/rs.3.rs-4997555/v1

Background: Suicide attempts are lethal and self-destructive behaviors that, in some cases, lead to death or irreversible physical harm and are associated with various factors. This study aimed to identify suicide-related death risk factors.

Materials and Methods: This study utilized data recorded in the suicide registry system of hospitals in Ilam Province. The data were analyzed via the Chi-square test in SPSS software. After the factors influencing suicide-related death were identified, their significance was evaluated and compared via logistic regression via the Python programming language and SPSS software. Subsequently, models for predicting suicide outcomes were developed via support vector machine (SVM), logistic regression, K-nearest neighbors (KNN), decision tree (DT), and random forest (RF) methods. These models were compared based on accuracy, recall, F1 score, and area under the receiver operating characteristic (ROC) curve.

Results: Among 3,833 cases of suicide in various hospitals in Ilam Province, the results indicated that the method of suicide (P<0.001), reason for attempting suicide (P<0.001), age group (P<0.001), education level (P<0.001), marital status (P=0.008), and employment status (P=0.002) were significantly associated with suicide-related death. Variables such as the season of suicide attempt, sex, father's education level, and mother's education level were not significantly related to suicide-related death. Furthermore, the random forest model demonstrated the highest area under the ROC curve (0.79) and the highest classification accuracy and F1 score on both the training data (0.85 and 0.2, respectively) and test data (0.87 and 0.22, respectively) for predicting suicide outcomes among the models tested.

Conclusion: This study revealed that older age, lower education level, divorce or widowhood, retirement, the use of physical methods and tools for suicide, and socioeconomic problems were significantly associated with suicide-related deaths. Additionally, the random forest model showed the best performance among the models tested in predicting suicide outcomes when these factors were used.

Suicide

Suicide Attempt

Suicide-related Death

Suicide Risk Factors

Machine Learning

Classification Algorithms

Natural factors and diseases were the primary causes of mortality in the past. However, many of these causes have become easily preventable and treatable with advancements in science and medicine. In recent decades, significant changes have occurred in the causes of death, with statistics indicating that nearly one million people worldwide die by suicide each year (1).

In the 19th century, suicide and attempted suicide were considered crimes in many countries around the world. However, in the 20th century, most of these laws were repealed. The reason for these widespread changes is that suicide and similar behaviors are now more often considered as symptoms of mental disorders and public health problems rather than as crimes or offenses (2).

Globally, suicide is the second leading cause of death among individuals aged 15 to 29 years (after traffic accidents) and the third leading cause of death in the 15 to 44 age group. The suicide rate varies across different countries, typically measured as the number of suicides per 100,000 people. The lowest rates range from 0 to 4.9. Tragically, in 2015, the vast majority—78%—of all suicides occurred in low- and middle-income countries (LMICs) (1, 3).

In Iran, it was estimated that in 2000, 6.6 per 100,000 people committed suicide. However, this figure has increased to 9.9 per 100,000 people in the last two decades. Additionally, according to a conservative estimate, over 198,000 people in Iran attempted suicide in 2016. Thus, expanding knowledge and strategies for suicide prevention is deemed essential in Iranian society (4).

In recent years, the use of machine learning models has increased in various fields, including healthcare, due to the absence of certain limitations in traditional classification models. In predicting the risk of suicide and the resulting death, machine learning models have performed better in most cases compared to theory-based models (5, 6).

Classification algorithms are a category of machine learning algorithms aimed at predicting which category each data point belongs to. For instance, an algorithm that classifies tumors into benign and malignant groups would be a classification algorithm. Typically, these algorithms calculate the probability that each data point belongs to a given class with a value between 0 and 1. The algorithm assigns each data point to the class with this probability exceeding a specific threshold. In binary classification problems, this decision threshold is usually 0.5, but it can vary depending on the problem's conditions and the data characteristics (7).

In the present study, we aimed to identify the factors influencing suicide-related death and then use different classification algorithms to predict the outcome of suicide attempts. Finally, we compared the performance of these algorithms.

Study Population

The study population consisted of all recorded cases of suicide and attempted suicide in Ilam Province from 2017 to 2022. Data were collected through ongoing event registration in the Ilam Province hospitals and were entered into a registry system.

Evaluation of Variables

The necessary data for this study were collected from the registry system based on the study's objectives and variables. Duplicate data were removed, and data were coded for entry into the analytical software using Python programming language version 3.11. In the study's first phase, only data from 2020 to the end of 2022 were used. In this phase, we selected ten variables based on the results and recommendations of previous related studies. We examined their association with the outcome of suicide attempts using the Chi-square test in SPSS software (8–11). The variables included in this phase were age group, gender, marital status, season, method of suicide, reason for suicide, education level, occupation, father's education level, and mother's education level. It is important to note that the significance level for this phase was set at (P < 0.05).

Design and Validation of Machine Learning Models

In the next phase, we used the variables that were associated with the outcome of suicide in the first phase to predict the outcome. Given that splitting the data used in the previous phase into training and testing sets could lead to overfitting, and some variables reported to be associated with suicide outcomes in previous studies (such as weather conditions or holidays at the time of the suicide attempt) were linked to the season of the suicide attempt but were not present in our dataset (12, 13), we used the data from the previous phase as the training data and the data from 2017 to the end of 2019 as the test data. In this phase, we used Python programming language and the Scikit-Learn library to design various models for predicting suicide outcomes. The algorithms used for modeling in this phase included Decision Tree (14), K-Nearest Neighbors (15), Logistic Regression (16), Random Forest (17), and Support Vector Machine (18). Additionally, to find the best hyperparameters for each algorithm, we employed a randomized search with 5-fold cross-validation and 200 iterations (19). Finally, the best performance of each algorithm, in terms of the Area Under the Receiver Operating Characteristic Curve (AUC), was selected as the model representative for that algorithm to be compared with the other algorithms (20).

Comparison of Classification Algorithms

Finally, the classification algorithms used in the second phase were compared using metrics such as Area Under the Receiver Operating Characteristic Curve (AUC), Accuracy, Recall, Precision, and F1 Score to determine the best classification algorithm for predicting the outcome of suicide attempts.

Overall, this study examined data from 4,013 cases of suicide attempts recorded in the Ilam Province registry system during the study period. Among these, 180 duplicate cases were removed, leaving 3,833 cases for analysis. It should be noted that patients who attempted suicide and their companions often do not have the physical and mental condition to cooperate adequately with the interviewer. Additionally, due to potential errors by the interviewer or data entry personnel, some information about each patient is often incompletely recorded in the registry system. Table 1 details the missing data for key variables recorded in the registry system for the 3,833 cases analyzed in this study. It should be noted that only cases with complete data for each variable were included in the statistical analysis, and cases with missing data were excluded.

Table 1

Frequency of Complete and Missing Data for Each Variable
Variable	Complete		Missing
Variable	Frequency	Percentage	Frequency	Percentage
Age group	3833	100.0%	0	0.0%
Gender	3833	100.0%	0	0.0%
Marital Status	3794	99.0%	39	1.0%
season	3833	100.0%	0	0.0%
Suicide Method	3691	96.3%	142	3.7%
Reason for Suicide	1470	38.4%	2363	61.6%
Education	3467	90.5%	366	9.5%
Occupation	3550	92.6%	283	7.4%
Father's Education	610	15.9%	3223	84.1%
Mother's Education	600	15.7%	3233	84.3%

Additionally, another problem related to the data of this study was the discrepancy in the mortality statistics from suicide between the hospital registry system and the forensic medicine data (Fig. 1). This discrepancy arose because many suicide victims died at the scene before reaching the hospital. As a result, these cases were transferred directly to forensic medicine without being recorded in the hospital registry. However, due to the lack of individual-level data in the information provided by the forensic department, the hospital registry system remains the best option for analyzing suicide-related data.

Suicide resulting in death was significantly associated with age group (P < 0.001), education level (P < 0.001), marital status (P = 0.008), occupation (P = 0.002), suicide method (P < 0.001), and the reason for the suicide attempt (P < 0.001). However, no significant association was found with the season of the suicide attempt (P = 0.62), gender (P = 0.19), father’s education level (P = 0.727), or mother’s education level (P = 0.421) (Table 2).

Table 2

Frequency of Variables and Chi-Square Test Results
	Suicide Outcome
	Non-Fatal	Fatal	Total	P-Value
Age Group				0.001>
5 to 14 years	129	4	133
15 to 24 years	1707	43	1750
25 to 34 years	1044	35	1079
35 to 44 years	531	23	554
45 to 54 years	178	10	188
55 to 64 years	57	9	66
65 years and above	57	6	63
Education				0.001>
Illiterate	94	12	106
Primary	158	9	167
Middle School	590	17	607
High School	2067	54	2121
University	451	15	466
Marital Status				0.008
Single	2094	60	2154
Married	1541	64	1605
Widowed	2	0	2
Divorced	29	4	33
Season				0.62
Spring	1070	36	1106
Summer	931	36	967
Autumn	781	30	811
Winter	917	26	943
Gender				0.19
Male	1806	71	1877
Female	1897	59	1956
Occupation				0.002
Unemployed	821	29	850
Housewife	1028	36	1064
Self-Employed	667	26	693
Government Employee	83	9	92
Retired	6	1	7
Student	781	15	796
Military Service	47	1	48
Father's Education				0.727
Illiterate	300	12	312
Primary	71	3	74
Middle School	80	1	81
High School	80	3	83
University	59	1	60
Mother's Education				0.421
Illiterate	331	11	342
Primary	80	4	84
Middle School	66	0	66
High School	75	1	76
University	31	1	32
Suicide Method				0.001>
Chemical	3427	82	3509
Physical	148	34	182
Reason for Suicide				0.001>
Psychological Issues	530	21	551
Physical Illness	20	2	22
Family Disputes	794	27	821
Socioeconomic Problems	66	10	76

After identifying significant variables using the Chi-square test, these variables were retained for modeling with machine learning classification methods, and the remaining variables were discarded. Additionally, in some variables, some categories that had less data and related to each other were merged to enhance the validity of the analysis. Table 3 shows the final categories of the variables used in the classification models and the odds ratio for each category.

Table 3. Results of univariate logistic regression on significant variables in the first phase

Next, various indices for comparing classification models are examined. The first index to be reviewed is classification accuracy. The Random Forest algorithm performed best in training and test data sets, with a value of 0.85 for the training data and 0.87 for the test data (Fig. 2).

The next index that was checked was classification precision. The highest value of this index in the training data belonged to the Logistic Regression and Random Forest algorithms (both 0.12). In the test data, it belonged to the Random Forest algorithm (0.22) (Fig. 3).

Another index to be reviewed is recall. The highest value of this index in both the training and test data sets was related to the Decision Tree algorithm (0.61 and 0.66, respectively). The Random Forest algorithm had the best values in most indices and the lowest values in this index (0.52 in the training data and 0.58 in the test data) (Fig. 4).

In cases like the present study, where the two previous indices show conflicting results, the F1 score, which is the geometric mean of these two indices, can be helpful. The Random Forest and Logistic Regression algorithms had the highest F1 score (0.2) in the training data. The highest value of this index in the test data was also related to the Random Forest algorithm (0.32) (Fig. 5).

The Receiver Operating Characteristic (ROC) curve and the area under it are other metrics that can be used to compare the quality of classification algorithms. This value was highest in the Random Forest and K-Nearest Neighbors algorithms (0.79) (Fig. 6).

This study examined the most significant factors influencing suicide mortality in Ilam Province. Globally, approximately 800,000 individuals die by suicide each year, which constitutes 1.5% of all deaths. Suicide is the tenth leading cause of death in North America and the primary cause of death for individuals aged 15 to 24 worldwide (21). Most individuals who survive a suicide attempt do not die from subsequent suicide attempts. This suggests that preventing methods leading to death could save many lives (22, 23).

Our research found that the highest likelihood of death following a suicide attempt occurred in individuals aged 55 to 65 years and those over 65 years. A significant association was observed between age and suicide mortality. These findings align with studies by Beghi M et al. (24)and Chen IM et al. (25), which identified advanced age as a critical risk factor for fatal suicide attempts. This may be related to comorbid conditions and fewer attempts involving non-lethal, attention-seeking methods among older adults.

Haghparast-Bidgoli H et al. (26) reported that an increase in the average educational level of populations in Iranian provinces significantly reduced suicide mortality rates. Similarly, our study found the highest risk of suicide death in illiterate individuals, followed by those with only primary education. A significant relationship was observed between education level and suicide mortality, consistent with the study mentioned above and other research by Favril L et al. (9) and Denney JT et al. (27).

Yoshimasu K et al. (28) described a minimal relationship between marital status and suicide mortality. However, our study, along with Haghparast-Bidgoli H et al. (26), demonstrated a significant association between divorce and suicide mortality. Denney JT et al. (27) also found a link between divorce and suicide mortality in men, with no similar relationship observed in women. These differences may be rooted in the cultural context and the stigma associated with divorce in Iranian society.

Although our study did not find a significant association between suicide mortality and different seasons of the year, the highest number of fatal suicides occurred in the spring and summer months. Previous studies by Villeneuve PJ et al. (12) and Likhvar V et al. (13) showed a correlation between warmer seasons and increased suicide mortality. Likhvar V et al. (13) also noted an inverse relationship between holidays and suicide mortality rates.

Wu Y et al. (29) and Chen IM et al. (25) reported that suicide mortality is higher in men than in women. Similarly, our study found that suicide deaths and the probability of suicide mortality were higher in men compared to women. However, a significant relationship between gender and suicide mortality was not observed. Denney JT et al. (27) assessed the impact of various factors on suicide mortality by gender, revealing differences in risk factors between men and women, which may underscore the importance of gender in determining suicide-related factors.

Yoshimasu K et al. (28) found minimal connections between social factors such as marital status and employment status and suicide mortality. In contrast, our study showed the highest likelihood of suicide deaths among retirees, and a significant relationship was found between employment status and suicide mortality. Moreover, studies by Graetz N et al. (30), Denney JT et al. (27), Haghparast-Bidgoli H et al. (26), and Favril L et al. (9) identified unemployment and lack of employment as risk factors for suicide mortality. These discrepancies may stem from differences in job classification between our study and others.

Vidal-Ribas et al. (31) found a direct relationship between low parental education levels and suicide mortality. In our study, although a significant relationship between parental education and suicide mortality was not observed, the highest rates of suicide were found among individuals with parents who had only primary education or no education, consistent with the previous study.

Cai Z et al. (32) identified firearms, hanging, and drowning as the most lethal suicide methods. Elnour AA and Harrison J (33) similarly ranked firearms and hanging as the most fatal methods. Although our study did not analyze suicide methods separately, it compared physical and chemical methods and found that physical methods resulted in higher mortality rates, aligning with previous research.

Favril L et al. (9) reported that psychiatric disorders, physical illnesses, and socioeconomic factors significantly impact suicide mortality rates. Our study also found a significant relationship between the cause of suicide attempts and mortality rates, consistent with the study mentioned above.

We aimed to design models for predicting suicide outcomes using five standard machine-learning classification methods based on variables initially associated with suicide outcomes. Among these, Random Forest and K-Nearest Neighbors models achieved the highest Area Under the Curve (AUC) value of 0.79. In test data, Random Forest demonstrated the highest classification accuracy (0.87), precision (0.22), and F1 score (0.32). The Decision Tree algorithm had the highest classification recall (0.66). For training data, Random Forest also showed the highest classification accuracy (0.85), while it, along with Logistic Regression, achieved the highest precision (0.12) and F1 score (0.20). Nevertheless, the Decision Tree had the highest classification recall (0.61) in training and testing data. Previous research by Amini et al. (34) comparing various models, including Logistic Regression, Support Vector Machine, Decision Tree, and Artificial Neural Networks, found the Support Vector Machine to have the best performance with an accuracy rate of 0.68, sensitivity of 0.85, and specificity of 0.67 in predicting suicide outcomes using training data. The positive predictive values in the different models used in this research were between 0.16 and 0.22. In contrast, in the study of Belsher BE et al. (35), the value of this index was reported to be less than 0.01 in most of the previous studies in the field of suicide; this finding is also consistent with the study of Schafer KM et al. (6).

In this study, we investigated the factors influencing suicide and, after identifying these factors, compared the predictive performance of five different machine learning models for suicide outcomes. The variables of marital status, education level, age group, occupation, reason for attempting suicide, and method of suicide were found to be significant predictors of suicide outcomes. Among the models evaluated, the Random Forest algorithm best predicted suicide outcomes using these variables.

Machine learning models, and especially models based on the random forest algorithm, can be used for the initial estimation of the condition of a person attempting to commit suicide when they go to a medical center. Further research is needed to validate the effectiveness of these models using more extensive datasets and additional variables and explore the potential of other algorithms, including hybrid algorithms, in this domain.

Ethics approval and consent to participate

The ethics committee of Ilam University of Medical Sciences approved the thesis from which this study was extracted. The ethics number was "IR.MEDILAM.REC.1401.179".

Clinical trial number: not applicable.

Consent for publication

Not applicable

Availability of data and materials

The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.

Competing interests

The authors declare that they have no conflict of interest.

Funding

This study had no funding.

Authors' contributions

M.B. cleaned and analyzed data and wrote the manuscript. K.S. assisted in the study design and data collection and supervised the procedures. Both authors read and approved the final manuscript.

Acknowledgments

The Ilam University of Medical Science has supported this study. The manuscript is extracted from the first author's professional doctorate thesis.

Authors' information

Bazarafshan M is a general physician and an Ilam University of Medical Sciences graduate. He presented the idea of this project due to his activity in Python programming and his interest in expanding the use of machine learning in medicine. Sayehmiri K. is a professor of statistics and is currently working in the Department of Statistics, Faculty of Health, Ilam University of Medical Sciences. He has a long history of research in various fields, including psychiatry and especially suicide.

Contributor Information

Mohammad Bazrafshan: Email: [email protected]

Kourosh Sayehmiri: Email: [email protected]

Organization WH. Suicide worldwide in 2019: global health estimates. 2021.
Wu KC-C, Cai Z, Chang Q, Chang S-S, Yip PSF, Chen Y-Y. Criminalisation of suicide and suicide rates: an ecological study of 171 countries in the world. BMJ open. 2022;12(2):e049425.
Bachmann S. Epidemiology of suicide and the psychiatric perspective. International journal of environmental research and public health. 2018;15(7):1425.
Kiani Chalmardi A, Rashid S, Honarmand P, Tamook F. A structural test of the interpersonal theory of suicide model in students. Contemporary Psychology, Biannual Journal of the Iranian Psychological Association. 2018;13(1):50-61.
Lee W, Lee J, Woo S-I, Choi SH, Bae J-W, Jung S, et al. Machine learning enhances the performance of short and long-term mortality prediction model in non-ST-segment elevation myocardial infarction. Scientific reports. 2021;11(1):12886.
Schafer KM, Kennedy G, Gallyer A, Resnik P. A direct comparison of theory-driven and machine learning prediction of suicide: A meta-analysis. PloS one. 2021;16(4):e0249833.
Sidey-Gibbons JA, Sidey-Gibbons CJ. Machine learning in medicine: a practical introduction. BMC medical research methodology. 2019;19:1-18.
Nie J, O'Neil A, Liao B, Lu C, Aune D, Wang Y. Risk factors for completed suicide in the general population: a prospective cohort study of 242, 952 people. Journal of affective disorders. 2021;282:707-11.
Favril L, Yu R, Geddes JR, Fazel S. Individual-level risk factors for suicide mortality in the general population: an umbrella review. The Lancet Public Health. 2023;8(11):e868-e77.
Favril L, Yu R, Uyar A, Sharpe M, Fazel S. Risk factors for suicide in adults: systematic review and meta-analysis of psychological autopsy studies. BMJ Ment Health. 2022;25(4):148-55.
Chau K, Kabuth B, Chau N. Gender and Family Disparities in Suicide Attempt and Role of Socioeconomic, School, and Health‐Related Difficulties in Early Adolescence. BioMed research international. 2014;2014(1):314521.
Villeneuve PJ, Huynh D, Lavigne É, Colman I, Anisman H, Peters C, et al. Daily changes in ambient air pollution concentrations and temperature and suicide mortality in Canada: Findings from a national time-stratified case-crossover study. Environmental Research. 2023;223:115477.
Likhvar V, Honda Y, Ono M. Relation between temperature and suicide mortality in Japan in the presence of other confounding factors using time-series analysis with a semiparametric approach. Environmental health and preventive medicine. 2011;16:36-43.
Quinlan JR. Induction of decision trees. Machine learning. 1986;1:81-106.
Altman NS. An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician. 1992;46(3):175-85.
Stoltzfus JC. Logistic regression: a brief primer. Academic emergency medicine. 2011;18(10):1099-104.
Breiman L. Random forests. Machine learning. 2001;45:5-32.
Cortes C, Vapnik V. Support-vector networks. Machine learning. 1995;20:273-97.
Bergstra J, Bengio Y. Random search for hyper-parameter optimization. Journal of machine learning research. 2012;13(2).
Ling CX, Huang J, Zhang H, editors. AUC: a better measure than accuracy in comparing learning algorithms. Advances in Artificial Intelligence: 16th Conference of the Canadian Society for Computational Studies of Intelligence, AI 2003, Halifax, Canada, June 11–13, 2003, Proceedings 16; 2003: Springer.
Ropper AH. Seena Fazel, MD, and Bo Runeson, MD, Ph. D. N Engl J Med. 2020;382:266-74.
Daigle MS. Suicide prevention through means restriction: Assessing the risk of substitution: A critical review and synthesis. Accident Analysis & Prevention. 2005;37(4):625-32.
Yip PS, Caine E, Yousuf S, Chang S-S, Wu KC-C, Chen Y-Y. Means restriction for suicide prevention. The Lancet. 2012;379(9834):2393-9.
Beghi M, Rosenbaum JF, Cerri C, Cornaggia CM. Risk factors for fatal and nonfatal repetition of suicide attempts: a literature review. Neuropsychiatric disease and treatment. 2013:1725-36.
Chen I-M, Liao S-C, Lee M-B, Wu C-Y, Lin P-H, Chen WJ. Risk factors of suicide mortality among multiple attempters: a national registry study in Taiwan. Journal of the Formosan Medical Association. 2016;115(5):364-71.
Haghparast-Bidgoli H, Rinaldi G, Shahnavazi H, Bouraghi H, Kiadaliri AA. Socio-demographic and economics factors associated with suicide mortality in Iran, 2001–2010: application of a decomposition model. International journal for equity in health. 2018;17(1):1-7.
Denney JT, Rogers RG, Krueger PM, Wadsworth T. Adult suicide mortality in the United States: marital status, family size, socioeconomic status, and differences by sex. Social science quarterly. 2009;90(5):1167-85.
Yoshimasu K, Kiyohara C, Miyashita K, Hygiene SRGotJSf. Suicidal risk factors and completed suicide: meta-analyses based on psychological autopsy studies. Environmental health and preventive medicine. 2008;13:243-56.
Wu Y, Schwebel DC, Huang Y, Ning P, Cheng P, Hu G. Sex-specific and age-specific suicide mortality by method in 58 countries between 2000 and 2015. Injury Prevention. 2020.
Graetz N, Preston SH, Peele M, Elo IT. Ecological factors associated with suicide mortality among non-Hispanic whites. BMC public health. 2020;20:1-12.
Vidal-Ribas P, Govender T, Sundaram R, Perlis RH, Gilman SE. Prenatal origins of suicide mortality: A prospective cohort study in the United States. Translational psychiatry. 2022;12(1):14.
Cai Z, Junus A, Chang Q, Yip PS. The lethality of suicide methods: A systematic review and meta-analysis. Journal of affective disorders. 2022;300:121-9.
Elnour AA, Harrison J. Lethality of suicide methods. Injury Prevention. 2008;14(1):39-45.
Amini P, Ahmadinia H, Poorolajal J, Amiri MM. Evaluating the high risk groups for suicide: a comparison of logistic regression, support vector machine, decision tree and artificial neural network. Iranian journal of public health. 2016;45(9):1179.
Belsher BE, Smolenski DJ, Pruitt LD, Bush NE, Beech EH, Workman DE, et al. Prediction models for suicide attempts and deaths: a systematic review and simulation. JAMA psychiatry. 2019;76(6):642-51.

No competing interests reported.

Predicting Suicide Outcomes: An Analysis of Key Factors and Machine Learning Models

Status:

Version 1

Abstract

Figures

Background

Methodology

Study Population

Evaluation of Variables

Design and Validation of Machine Learning Models

Comparison of Classification Algorithms

Results

Discussion

Conclusion

Declarations

References

Additional Declarations

Status:

Version 1