The influence of anti-involution training on the critical thinking of young healthcare professionals in dental outpatient clinics-based on machine learning model

doi:10.21203/rs.3.rs-3908847/v1

Background

The relationship between the impact of anti-involution training on critical thinking and its propensity indicators among young healthcare professionals in dental outpatient clinics remains to be determined. Therefore, this study aimed to investigate these associations and develop an interpretable machine learning (ML) model to assess their predictive value in enhancing critical thinking through anti-involution training.

Methods

A cross-sectional survey encompassing 114 participants was conducted. Spearman correlation analysis was utilized to evaluate the association between propensity indicators and the enhancement of critical thinking through anti-involution training. Subsequently, the data underwent normalization utilizing the “MinMaxScaler” technique, while balancing was achieved by applying the synthetic minority oversampling technique (SMOTE). Following this, predictors were identified using the most minor absolute shrinkage and selection operator (LASSO) regression. Next, diverse machine learning algorithms constructed an individual prediction model to enhance critical thinking through anti-involution training. The prediction model's performance was assessed using receiver operating characteristic (ROC) curve analysis and decision curve analysis (DCA). The Shapley additive interpretation (SHAP) method was utilized to interpret the ML model.

Results

Truth-seeking, analytical thinking, and inquisitiveness were identified as predictive factors for enhancing critical thinking. A Random Forest algorithm-based model incorporating these variables yielded favorable results: AUC = 0.889 (95% CI: 0.839–0.937), accuracy = 0.850, sensitivity = 0.855, specificity = 0.933.

Conclusion

The inclinations toward truth-seeking, analytical thinking, and inquisitiveness significantly correlate with the effectiveness of anti-involution training in enhancing critical thinking. Our simplified ML-based predictive model allows for preliminary forecasting, enabling early intervention and guidance for learners facing difficulties in improving critical thinking.

anti-involution training

Critical thinking

Propensity indicators

Machine learning

Critical thinking is a rational and reflective mode manifested when confronted with evidence, situations, criteria, and concepts (Sari et al., 2021). As healthcare professionals are intricately involved in the lives of individuals, there is often a need to make judicious, prompt, and rational judgments based on professional knowledge and the best available evidence, coupled with clinical realities and patient preferences. This necessitates the possession of providers to minimize errors and optimize clinical decision-making (Dissen, 2023).

The phenomenon of involution, which has been highly scrutinized in recent years by scholars, is a complex, evolving, and multidimensional concept. It refers to the situation where individuals within a profession exert increasing efforts to compete for limited resources, decreasing the individual’s "benefit-to-effort ratio." This state is characterized by"internal consumption" or "irrational competition" (Dou et al., 2022; McCullough, 2019). The high level of involution in a societal structure generates social anxiety, negatively impacting the development of various professions (Chen et al., 2022; D. Yi et al., 2022). Therefore, it becomes necessary for all professions to strive towards anti-involution. In the healthcare industry, a phenomenon known as involution also exists among healthcare professionals, which hurts their professional growth and the delivery of medical services. This is particularly evident in medical research, where involution is pronounced. Although numerous research achievements are produced, they contribute little to advancing the research field, as scientific inquiry often remains confined to a simplistic level of self-depletion and repetition (Gan, 2023). Therefore, it is crucial to provide anti-involution training for healthcare professionals.

Cultivating anti-involution and critical thinking is essential in developing healthcare professionals, and exploring the relationship between them is vital in their training. Research indicates that both anti-involution training and critical thinking are associated with innovative thinking (Handayani & Wulandari, 2021; Mu, 2021; Zeng, 2023). It can be shown that there is an indirect correlation between anti-involution training and critical thinking. Still, no direct correlation between the two has been reported in the literature.The effects of anti-involution training, young healthcare professionals in dental outpatient clinics, and enhancing critical thinking can be understood as components of the teaching process, learners, and learning outcomes. Existing studies have shown that factors influencing learner performance are multifaceted and can be classified into two categories of indicators (Al-Rahmi & Zeki, 2017; Mosimege & Winnaar, 2021; Uwineza et al., 2023). The first category refers to predispositional indicators, representing the attributes learners possess upon entering the learning environment (static indicators). The second category consists of behavioral performance indicators, which reflect learners’ performance during the learning process (dynamic indicators) (Efthimiopoulos & Mylonakou-Keke, 2012; Magsino, 2021; Müller & Seufert, 2018; Nickl et al., 2022). To date, no studies have reported on the factors influencing the effectiveness of anti-involution training in improving critical thinking.

In this study, we aim to explore the correlation between anti-involution training and propensity indicators of young healthcare professionals in dental outpatient clinics in relation to the enhancement of critical thinking. Using machine learning algorithms, we will identify the key factors influencing the effectiveness of anti-involution training in improving critical thinking and construct a machine learning model to quantify the impact of these factors. This will enable educational professionals to intervene and guide learners at risk of poor learning performance in advance, thus maximizing the improvement of critical thinking among young healthcare professionals in dental outpatient clinics through anti-involution training. Additionally, we will utilize the SHAP method to enhance the Interpretability of the model, ensuring that the results are easily understood and applicable.

Participants

This cross-sectional study was conducted in Kunming, China, from January to December 2020. The study participants comprised young healthcare professionals under 40 from five outpatient departments of the Affiliated Stomatological Hospital of Kunming Medical University. One hundred fourteen individuals participated in the anti-involution training before the study, while 91 individuals participated after the training. A total of 205 questionnaires were returned, and after excluding cases with missing essential information and obvious logical errors, 182 questionnaires were deemed suitable for analysis, resulting in an effective response rate of 88.78%. Informed consent was obtained from all study participants.

Data collection

The Chinese version of the Critical Thinking Disposition Inventory (CTDI-CV) was utilized as the survey tool in this study. The overall Cronbach’s α coefficient of the CTDI-CV was 0.90. The inventory consists of positive and negative items, rated on a 6-point Likert scale. The scoring for positive items ranges from 1 (strongly disagree) to 6 (strongly agree), while the scoring for negative items is the reverse (Zhang et al., 2017). The CTDI-CV encompasses seven traits comprising 70 items: truth-seeking, analyticity, open-mindedness, systematicity, inquisitiveness, self-confidence in critical thinking, and cognitive maturity. Each trait includes ten items, resulting in a total score range of 420 (Huang et al., 2021).

Before completing the questionnaires, the research team provided detailed explanations to the participants regarding the purpose, significance, instructions, precautions, and deadline for questionnaire submission. Before the anti-involution training, baseline measurements were taken from young healthcare professionals under 40 in five outpatient departments of the Affiliated Stomatological Hospital of Kunming Medical University. The training, led by the department directors, lasted for one year and included a series of lectures on involution-related theories. Regular assessments on involution-related knowledge points and measurements of critical thinking disposition were conducted every three months.

The data collected in this study encompassed two groups of factors: (1) participants’ propensity indicators, including initial knowledge and skills and inherent indicators. Initial knowledge and skills involved the pre-training total score of CTDI-CV, as well as the total scores of the seven traits of CTDI-CV before the training, including truth-seeking, analyticity, open-mindedness, systematicity, inquisitiveness, self-confidence in critical thinking, and cognitive maturity. Inherent indicators included gender and profession. (2) Conclusion: The effects of anti-involution training on enhancing critical thinking among young healthcare professionals in dental outpatient clinics. The difference between the two total scores of CTDI-CV was used to measure the effects of anti-involution training on enhancing critical thinking among young healthcare professionals in dental outpatient clinics. A difference ≥ 25 indicated a significant improvement, while a difference < 25 indicated no significant improvement. Based on this criterion, the participants were divided into the significantly improved and non-significantly improved groups.

statistical analysis

Analytical methods

Descriptive analysis was conducted using SPSS (version 26.0), while other statistical analysis procedures were performed using R (version 3.6.1) and Python (version 3.4.3). Mann-Whitney U or Chi-square tests were used to analyze the demographic differences between the significantly improved and non-significantly improved groups, with continuous variables represented as median and interquartile range (IQR) and compared using the Mann-Whitney U test. Categorical variables were presented as counts and percentages and compared using the Chi-square test. Paired samples t test was used to analyze the characteristics of the effects of anti-involution training on critical thinking among young healthcare professionals in dental outpatient clinics. Spearman correlation analysis and multivariate logistic regression were employed to evaluate the relationships between predispositional indicators and the effects of anti-involution training on critical thinking among young healthcare professionals in dental outpatient clinics, as well as the influence of the former on the latter. A two-sided p-value less than 0.05 was considered statistically significant.

Data preprocessing

In order to mitigate the influence of indicator dimensions and magnitudes on the study results and to facilitate comparability of data across each test index, the test indicators were subjected to normalization. The normalization process employed in this study was "MinMaxScaler," which linearly transformed the original data to ensure that the resulting values fell within the range of [0, 1]. Additionally, data balancing was performed using the “SMOTE” technique, which equalized the proportion of minority and majority classes to a ratio of 1:1 (Nguyen et al., 2023).

Variable selection and prediction model establishment

We employed Lasso regression for feature selection to select the most predictive and relevant features to improve model performance and reduce computational burden (Bainter et al., 2023). Lasso regression compresses the coefficients of variables in the regression model by generating a penalty function to prevent overfitting and address severe collinearity issues (Genç & Özkale, 2022; Guler & Guler, 2021). Lasso regression was performed with 10-fold cross-validation using the “glmnet 4.1.2” R package. The study participants were divided into training and validation sets at 7:3. On the training set, we fitted nine machine learning classifiers: logistic regression, XGBoost, LightGBM, random forest, AdaBoost, Gaussian Naive Bayes (GNB), Support Vector Machine (SVM), multilayer perceptron (MLP), and K-nearest neighbors (KNN). XGBoost was built using the “xgBoost 1.2.1” Python package, while LightGBM was built using the “lightgbm 3.2.1” Python package. The other machine-learning algorithms were built using the “sklearn 0.22.1” Python package.

Model performance evaluation

Model performance was evaluated based on its discriminative, calibration, clinical applicability, and generalizability. Resampling was employed to ensure consistency of training samples across multiple models, enabling comparison of the results. The best model selection was based on analyzing the importance of metrics across models on both training and testing sets. The area under the curve (AUC), which is commonly used to characterize the diagnostic accuracy of a test or predictive model, was constructed using the “sklearn 0.22.1” Python package (Obuchowski & Bullen, 2018). Decision curve analysis (DCA) plots, representing a decision analysis technique with substantial advantages in assessing the clinical utility of models, were generated using the “rmda 1.6” R software (Vickers & Elkin, 2006). Calibration curves, which assess the predictive ability of models, were constructed using the “sklearn 0.22.1” Python package (Fenlon et al., 2018). Precision-recall (PR) curves, widely used in the field, were plotted using the “scikit 0.22.1” Python package to evaluate model performance (Li & Guo, 2021). Once the most helpful model was determined through comprehensive comparison, all eligible participants were reclassified into training, validation, and testing sets at 6:2:2. The best model underwent 5-fold cross-validation on the training set and was evaluated on the validation and testing sets. Learning curves were generated using the “scikit 0.22.1” Python package to assess the adaptability and stability of the model on the training and validation sets (Belkin et al., 2019).

Interpretability of the model

We utilized SHAP values to explain the optimal machine learning model. SHAP, derived from Shapley values in game theory, quantifies the contribution of each feature in the model to the final prediction. The SHAP summary plot combines feature importance and feature effects. Each point on the SHAP dependence plot represents a feature and its Shapley value for a specific instance. In SHAP explanations, feature attributes like Shapley values are visualized as "forces" that contribute to increasing or decreasing the prediction. Predictions are based on a baseline, and the baseline for Shapley values is computed as the average of all predictions. The "SHAP 0.39.0" Python package generated visual explanations of SHAP values for model importance and contributions (Futagami et al., 2021).

Characteristics of the study participants

The demographic characteristics of the participants are presented(Table 1) . In this study, we excluded a total of 23 participants with incomplete data, a total of 91 participants were enrolled in this study, with 62 participants who underwent anti-involution training showing no significant improvement in critical thinking skills and 29 participants showing significant improvement. Detailed data for 91 participants can be found in Supplementary Data S1. When comparing the non-significant improvement group to the significant improvement group, significant differences (p < 0.05) were observed in several critical thinking aspects, including CTDI-CV total score before training (p = 0.002), truth-seeking (p = 0.007), analyticity (p = 0.008), systematicity (p = 0.015), self-confidence in critical thinking (p = 0.047), inquisitiveness (p = 0.027), and cognitive maturity (p = 0.047). However, there were no significant differences (p > 0.05) in terms of profession (p = 0.378), gender (p = 0.953), and open-mindedness (p = 0.185).

Table 1 Characteristics in non-significantly improved group and significantly improved group

Variable	non-significantly improved group significantly (n=62)	significantly improved group(n=29)	p
profession,n(%)
Stomatology	24(38.710%)	10(34.483%)	0.378^b
Nursing	36(58.065%)	16(55.172%)
Other	2(3.226%)	3(10.345%)
gender
Male	11(17.742%)	5(17.241%)	0.953^b
female	51(82.258%)	24(82.759%)
CTDI-CV total score before training, median [IQR]	263.00[249.00,279.00]	244.00[212.00,256.00]	0.002^a
truth-seeking, median [IQR]	39.00[35.00,41.00]	33.00[30.00,40.00]	0.007^a
analyticity, median [IQR]	35.00[32.00,39.00]	31.00[28.00,36.00]	0.008^a
systematicity, median [IQR]	37.00[35.00,40.00]	33.00[30.00,37.00]	0.015^a
self-confidence in critical thinking, median [IQR]	35.00[31.00,40.00]	30.00[25.00,37.00]	0.047^a
inquisitiveness, median [IQR]	37.00[31.00,40.00]	33.00[27.00,38.00]	0.027^a
cognitive maturity, median [IQR]	40.00[36.00,47.00]	37.00[33.00,42.00]	0.047^a

p^avalues are calculated from the Mann-Whitney U test, and p^b was obtained by x²-tests. p < 0.05 indicates statistical significance. Numbers in bold mean statistical significance.

The effects of anti-involution training on the enhancement of critical thinking among young healthcare professionals in dental outpatient clinics

The effects of anti-involution training on the enhancement of critical thinking among young healthcare professionals in dental outpatient clinics are presented(Table 2 and Fig. 1). The results reveal that, among the seven characteristics of critical thinking, there is a slight decrease in the average score of self-confidence in critical thinking. The average scores of the remaining six characteristics and the total score have all shown slight improvements. Notably, there is a statistically significant difference in the average score increase for cognitive maturity, which is 3.978±-0.014 (P=0.004). Additionally, the average score increase for the total score is 11.813±6.752 (P=0.001), signifying a statistically significant difference.

Table 2 Scores of each critical thinking feature before and after anti-involution training（±s）

Variable	before anti-involution training (n=91)	After anti-involution training(n=91)	Increased scores	P
truth-seeking	36.989±6.918	38.132±6.843	1.143±-0.075	0.287
open-mindedness	37.308±6.458	38.341±6.469	1.033±0.011	0.343
analyticity	33.681±5.88	34.593±5.89	0.912±0.01	0.277
systematicity	35.198±7.14	36.121±7.082	0.923±-0.058	0.388
self-confidence in critical thinking	33.286±7.605	31.275±7.603	-2.011±-0.002	0.075
inquisitiveness	33.835±8.148	34.802±8.133	0.967±-0.015	0.427
cognitive maturity	39.637±8.846	43.615±8.833	3.978±-0.014	0.004^**
CTDI-CV total scores	249.934±39.513	261.747±46.266	11.813±6.752	0.001^***

* * * , * * , * represent 1% , 5% , 10% significance levels respectively. Numbers in bold mean statistical significance.

Correlation analysis of the effects of anti-involution training on critical thinking and propensity indicators among young healthcare professionals in dental outpatient clinics

As shown in Table 3, there is a negative correlation between truth-seeking, analyticity, systematicity, self-confidence in critical thinking, inquisitiveness, cognitive maturity, and CTDI-CV total scores with the effects of anti-involution training on the enhancement of critical thinking (truth-seeking: r = -0.284, p < 0.05; analyticity: r = -0.287, p < 0.05; systematicity: r = -0.258, p < 0.05; self-confidence in critical thinking: r = -0.210, p < 0.005; inquisitiveness: r = -0.234, p < 0.05; cognitive maturity: r = -0.210, p < 0.05; CTDI-CV total scores: r = -0.327, p > 0.05). However, there is no correlation between profession, gender, open-mindedness, and the effects of de-individuation training on the enhancement of critical thinking (profession: r = 0.078, p > 0.05; gender: r = -0.006, p > 0.05; open-mindedness: r =- 0.140, p > 0.05).

Table 3 The Spearman's association analysis among Tendency indicators and the effects of anti-involution training on critical thinking

Variable	The effects of anti-involution training on the enhancement of critical thinking
Variable	r	p
profession	0.078	0.461
gender	-0.006	0.954
truth-seeking	-0.284	0.006^**
open-mindedness	-0.140	0.185
analyticity	-0.278	0.008^**
systematicity	-0.258	0.014^**
self-confidence in critical thinking	-0.210	0.046^**
inquisitiveness	-0.234	0.026^**
cognitive maturity	-0.210	0.046^**
CTDI-CV total scores	-0.327	0.002^**

The p-values were calculated using the Spearman correlation analysis.* * * , * * , * represent 1% , 5% , 10% significance levels respectively. Numbers in bold mean statistical significance.

Identifying predictors

LASSO regression analysis was conducted on the remaining independent variables with a tophus as the dependent variable (Fig. 2). LASSO can compress variable coefficients to prevent overfitting and solve severe collinearity problems (F. Yi et al., 2023). The results showed that (lambda with minimum mean square error = 0.06) 10 independent variables were reduced to 4, including truth-seeking, analyticity, and inquisitiveness(Fig. 2a,b)

Comprehensive Analysis of Classified Multi-Mode

Multiple ML models were utilized for data sample classification: XGBoost, logistic regression, LightGBM, Random Forest, AdaBoost, KNN, SVM, GNB, and MLP. A 5-fold cross-validation approach was performed to validate all models. The AUC value was employed to evaluate model predictions (Obuchowski & Bullen, 2018). Results indicated that Random Forest, XGBoost, and AdaBoost exhibited superior performance on the training set, whereas Random Forest, KNN, and LightGBM achieved the highest performance on the validation set (Fig. 3a,b);see more details in Supplemental Table S1and Table S2.The AUC value primarily assesses the predictive accuracy of the models without providing information on their clinical utility or preferences (Muschelli, 2020; Obuchowski & Bullen, 2018). Therefore, we analyzed the PR curve, calibration curve, and DCA. The DCA curve illustrated the favorable clinical applicability of the Random Forest model (Fig. 3c). The calibration curve suggested that the Random Forest and KNN models provided more accurate predictions (Fig. 3d). The Random Forest model displayed exceptional performance in the training and validation sets, generating the highest average precision (AP) value on the validation set (Fig. 3e,f). Considering all factors, the comprehensive analysis suggests that Random Forest can be considered the optimal model.

The best model construction and evaluation

The Random Forest model was trained using the training set with 5-fold cross-validation. The results revealed that the average AUC for the training set was 0.912 (0.843-0.980), while for the validation set, it was 0.889 (0.740-0.990). The AUC for the test set was 0.868 (0.731-1.000) (Fig. 4a-c);see more details in Supplemental Table S3.Considering that the performance of the validation set did not exceed or differ by less than 10% from the test set based on the AUC metric, it can be concluded that the model was successfully fitted. The calibration curve further demonstrated that the Random Forest model was accurate and predictive (Fig. 4d). These outcomes suggest that the Random Forest model can be utilized for classification modeling tasks on the dataset.

The SHAP to Interpret ML model

We employed SHAP to interpret the model to provide an intuitive explanation of the selected variables.

The distribution of results for each participant is illustrated (Fig. 5a).In each feature importance line, different colored dots represent the attribution of results by all participants. Red dots represent high likelihood values, while blue dots represent low likelihood values. The variables inquisitiveness, truth-seeking, and analyticity ability of participants before anti-involution training are all negatively correlated with the results.Three risk factors were ranked by evaluating the average absolute SHAP values(Fig. 5b). The x-axis represents the importance of the predictive model measured by the SHAP values. The feature importance of the Random Forest model, from high to low, was observed as inquisitiveness, truth-seeking, and analyticity.

Furthermore, we provide two typical examples to illustrate the Interpretability of the model. For one participant, there was no significant improvement observed in critical thinking, as indicated by a lower SHAP prediction score of 0.15 (Fig. 5c).In contrast, another participant exhibited significant improvement in critical thinking, as evidenced by a higher SHAP score of 0.95 (Fig. 5d).

Critical thinking is both a skill and a mindset. It refers to an individual’s ability to analyze and solve problems in complex environments using existing knowledge and experience and the capacity to make decisions based on reflection, analysis, and reasoning (Ennis, 2018). Strong critical thinking skills are a fundamental ability and quality that healthcare professionals should possess (ŽivkoviĿ, 2016). The findings of this study indicate that the overall level of critical thinking abilities among young dental outpatient healthcare workers is moderate to low, falling short of the criterion for a positive critical thinking tendency (Cisneros, 2009; Gupta et al., 2012). Therefore, there is still ample room for improvement.

Involvement is a state of internal consumption or irrational competition, and highly involved social structures hurt the development of various professions within society. The “internalization” of the medical industry has brought about negative consequences for the growth of healthcare professionals and the execution of medical activities, which may hinder improving critical thinking. Previous studies have shown that critical thinking and innovative thinking complement each other, and the key to anti-involution lies in enhancing innovative thinking (Arce-Saavedra & Blumen, 2022; Jing, 2021; Ramírez-Montoya et al., 2022; Sharma & Priyamvada, 2022). This study explores the effects of anti-involution training on enhancing critical thinking among young dental outpatient healthcare workers. The results indicate that, aside from a slight decrease in the average score for self-confidence in critical thinking, there were small but significant improvements in the average scores for the other six characteristics of critical thinking and the overall score. This suggests that anti-involution training has a specific positive effect on the critical thinking of healthcare professionals, providing additional evidence for the direct relationship between anti-involution training and critical thinking.

However, this study also has some limitations. Firstly, there are various methods of anti-involution training. The single training method used in this study can only cover some aspects of anti-involution training. Secondly, there were fluctuations in the participants throughout the study period, resulting in the loss of 25 participants. To achieve a one-to-one match between the pre-and post-training samples, the data of the lost participants were excluded, which may lead to an insufficient sample size. Additionally, the study lasted one year, during which confounding factors may have been present, potentially introducing errors in the experimental results.

As an instructional practice, improving the effectiveness of anti-involution training is a concern for scholars. The factors influencing learners’ performance are complex, and it is challenging to predict them based on a single factor. Previous research has shown a strong correlation between dispositional indicators and learners’ performance, particularly regarding initial knowledge, skills, and inherent indicators (Considine & Zappalà, 2002; Deakin Crick et al., 2015; Friedman & Mandel, 2011). This study explores the correlation between improving critical thinking among young dental outpatient healthcare workers through anti-involution training and their dispositional indicators. The correlation analysis results indicate a significant statistical significance between improving critical thinking and the pre-training total score, pre-training analytical ability, self-confidence, cognitive maturity, systematic thinking ability, and truth-seeking of critical thinking. These dispositional indicators are closely related to innovative thinking, thus further confirming the close relationship between innovative thinking and critical thinking, which aligns with the previous research conclusion of our research team (Syed Marzuki et al., 2020). However, apart from the factors in this study, dispositional indicators include other aspects such as family background, initial grades, expectations or satisfaction with education, and goal expectations (Considine & Zappalà, 2002; Deakin Crick et al., 2015; Friedman & Mandel, 2011). The relationship between these indicators and the improvement of critical thinking among young dental outpatient healthcare workers through anti-involution training requires further research for verification.

Traditionally, education research has been relatively limited regarding research models, relying primarily on small-scale data and traditional statistical methods. Research data is often obtained through questionnaires and self-reports, with relatively limited sample sizes. Moreover, traditional statistical analysis methods have limitations in revealing complex relationships between variables. The widespread application of machine learning methods in various fields has gained attention and usage in social science research (Macalli et al., 2021). Machine learning has significant advantages in studying factors that influence learning performance. It excels in handling large datasets and extracting potential underlying connections that traditional methods may overlook. Recently, scholars have begun to use supervised machine learning methods such as Support Vector Machines (SVM), Random Forests, Deep Neural Networks (DNN), XGBoost, etc., to address classification and prediction problems in the field of educational research (Vergaray et al., 2022).

In this study, the Random Forest algorithm was ultimately selected to construct a predictive model for the effect of anti-involution training on the improvement of critical thinking among young dental outpatient healthcare workers. The Random Forest model is a widely used machine learning model. An ensemble learning method combines multiple decision trees to make predictions. Each decision tree in the Random Forest is built independently using a random subset of the dataset and random features, which helps reduce overfitting and improve generalization (Rigatti, 2017). One of the key advantages of the Random Forest model is its ability to handle high-dimensional datasets and maintain good performance. It can handle large-scale datasets with numerous features without sacrificing accuracy (Malhotra & Karanicolas, 2020). The Random Forest model is known for its robustness and resistance to noise and outliers, making it suitable for various real-world applications. The Random Forest model can also estimate feature importance. By calculating the average decrease in impurity or information gain caused by each feature, it can identify the most influential features for prediction. This feature importance analysis can be valuable for feature selection and understanding the underlying relationships within the dataset.

Furthermore, the Random Forest model can handle missing data and maintain accuracy even when a significant portion is missing. It can utilize the available information from other features to fill in missing values and make reliable predictions (Antoniadis et al., 2021). The Random Forest model is computationally efficient and can be parallelized for faster training and prediction. It can process large datasets efficiently using multi-core processors and distributed computing frameworks. In summary, the Random Forest model is a robust machine-learning algorithm that combines the predictions of multiple decision trees to provide accurate and robust results (Su et al., 2022). Its ability to handle high-dimensional datasets, estimate feature importance, handle missing data, and parallelize the computation makes it a popular choice for various applications in machine learning and data analysis (Gao et al., 2020).

In this study, the Random Forest model exhibited favorable performance across various measures, particularly regarding AUC, accuracy, and sensitivity. However, the model showed lower specificity and cutoff threshold, which may be attributed to factors such as insufficient sample size, the presence of confounding factors that cannot be eliminated, significant collinearity among certain variables, and the exclusion of certain key variables. Future research can further improve these aspects, potentially enhancing the model's predictive capability.

In summary, the results of this study indicate a significant correlation between the truth-seeking, analytical, and inquisitiveness in the propensity indicators and the effectiveness of anti-involution training in enhancing critical thinking. Based on these findings, we have developed a simplified and replicable ML-based predictive model to make preliminary predictions on improving critical thinking. This model enables educators to make preliminary forecasts regarding the improvement of critical thinking, thereby facilitating timely intervention and guidance for learners who encounter challenges in enhancing their critical thinking abilities.

ML Machine learning

SMOTE Synthetic minority oversampling technique

LASSO Least absolute shrinkage and selection operator

ROC Receiver operating characteristic

DCA Decision curve analysis

SHAP Shapley Additive exPlanations

AUC Area under the curve

CTDI-CV The Chinese version of the Critical Thinking Disposition Inventory

IQR Interquartile range

GNB Gaussian Naive Bayes

SVM Support Vector Machine

MLP Multilayer perceptron

KNN K-nearest neighbors

PR Precision-recall

DNN Deep Neural Networks

Acknowledgments We thank all participants in this study. We also thank the Kunming Medical University Dental Hospital for its assistance in data collection.

Author contributions LZ and CY conceived and designed this study. CY, ZA, LL and YH were responsible for data acquisition, analysis, and interpretation. CY, ZJ, LJ and CT participated in writing the manuscript. CY, LZ, and RX helped revise the manuscript. All the authors have read and approved the final manuscript.

Funding This study was supported by The Kunming Medical University education reform Project [2021-JY-Y-036] and received the support of the National Natural Science Foundation of China (82360185).

Data availability The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Conflict of interests The authors have no relevant competing interests to declare.

Ethics approval and consent to participate The ethical approval for this study has been submitted to the Ethics Committee of Kunming Medical University. The committee reviewed this study and determined that ethical approval is not required as it complies with institutional guidelines and national laws and regulations. Thisstudy did not involve human clinical trials or animal experiments. Participation in the study is voluntary and verbal consent has been obtained from all participants. The confidentiality of participant responses has been ensured, and the data is only used for scientific purposes. To ensure complete anonymity, no data that may identify the interviewee was disclosed. In addition, this study was approved and supported by the Project Guidance Committee of Kunming Medical University.

Consent for publication Not applicable.

Al-Rahmi, W. M. & Zeki, A. M. (2017). A model of using social media for collaborative learning to enhance learners’ performance on learning. Journal of King Saud University - Computer and Information Sciences, 29(4). https://doi.org/10.1016/j.jksuci.2016.09.002
Antoniadis, A., Lambert-Lacroix, S. & Poggi, J. M. (2021). Random forests for global sensitivity analysis: A selective review. In Reliability Engineering and System Safety (Vol. 206). https://doi.org/10.1016/j.ress.2020.107312
Arce-Saavedra, B. J. & Blumen, S. (2022). Critical thinking, creativity, self-efficacy, and teaching practice in Peruvian teacher trainers. Revista de Psicologia (Peru), 40(1). https://doi.org/10.18800/psico.202201.020
Bainter, S. A., McCauley, T. G., Fahmy, M. M., Goodman, Z. T., Kupis, L. B. & Rao, J. S. (2023). Comparing Bayesian Variable Selection to Lasso Approaches for Applications in Psychology. Psychometrika, 88(3). https://doi.org/10.1007/s11336-023-09914-9
Belkin, M., Hsu, D., Ma, S. & Mandal, S. (2019). Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences of the United States of America, 116(32). https://doi.org/10.1073/pnas.1903070116
Chen, Y., Han, D., Liu, Z., He, W. & Li, X. (2022). The Phenomena of “Education Involution” and the Root Cause Cracking. Frontiers in Business, Economics and Management, 6(1). https://doi.org/10.54097/fbem.v6i1.2288
Cisneros, R. M. (2009). Assessment of critical thinking in pharmacy students. American Journal of Pharmaceutical Education, 73(4). https://doi.org/10.5688/aj730466
Considine, G. & Zappalà, G. (2002). The influence of social and economic disadvantage in the academic performance of school students in Australia. Journal of Sociology, 38(2). https://doi.org/10.1177/144078302128756543
Deakin Crick, R., Huang, S., Ahmed Shafi, A. & Goldspink, C. (2015). Developing Resilient Agency in Learning: The Internal Structure of Learning Power. British Journal of Educational Studies, 63(2). https://doi.org/10.1080/00071005.2015.1006574
Dissen, A. (2023). A critical issue: assessing the critical thinking skills and dispositions of undergraduate health science students. Discover Education, 2(1). https://doi.org/10.1007/s44217-023-00044-z
Dou, G., Li, G., Yuan, Y., Liu, B. & Yang, L. (2022). Structural Dimension Exploration and Measurement Scale Development of Employee Involution in China’s Workplace Field. International Journal of Environmental Research and Public Health, 19(21). https://doi.org/10.3390/ijerph192114454
Efthimiopoulos, A. & Mylonakou-Keke, I. (2012). INTERACTION BETWEEN HUMAN AND INFORMATIVE SYSTEMS IN PRIMARY EDUCATION WITHIN A SOCIOPEDAGOGICAL CONTEXT. 5TH INTERNATIONAL CONFERENCE OF EDUCATION, RESEARCH AND INNOVATION (ICERI 2012).
Ennis, R. H. (2018). Critical Thinking Across the Curriculum: A Vision. Topoi, 37(1). https://doi.org/10.1007/s11245-016-9401-4
Fenlon, C., O’Grady, L., Doherty, M. L. & Dunnion, J. (2018). A discussion of calibration techniques for evaluating binary and categorical predictive models. In Preventive Veterinary Medicine (Vol. 149). https://doi.org/10.1016/j.prevetmed.2017.11.018
Friedman, B. A. & Mandel, R. G. (2011). Motivation predictors of college student academic performance and retention. In Journal of College Student Retention: Research, Theory and Practice (Vol. 13, Issue 1). https://doi.org/10.2190/CS.13.1.a
Futagami, K., Fukazawa, Y., Kapoor, N. & Kito, T. (2021). Pairwise acquisition prediction with SHAP value interpretation. Journal of Finance and Data Science, 7. https://doi.org/10.1016/j.jfds.2021.02.001
Gan, N. (2023). The Effect of Cultural Capital on Students’ Academic Motivation in the Context of Involution: Ecological Perspective. SHS Web of Conferences, 163. https://doi.org/10.1051/shsconf/202316304014
Gao, K., Mei, G., Piccialli, F., Cuomo, S., Tu, J. & Huo, Z. (2020). Julia language in machine learning: Algorithms, applications, and open issues. In Computer Science Review (Vol. 37). https://doi.org/10.1016/j.cosrev.2020.100254
Genç, M. & Özkale, M. R. (2022). Lasso regression under stochastic restrictions in linear regression: An application to genomic data. Communications in Statistics - Theory and Methods. https://doi.org/10.1080/03610926.2022.2149243
Guler, H. & Guler, E. O. (2021). Mixed Lasso estimator for stochastic restricted regression models. Journal of Applied Statistics, 48(13–15). https://doi.org/10.1080/02664763.2021.1922614
Gupta, K., Iranfar, S., Iranfar, K., Mehraban, B. & Montazeri, N. (2012). Validly and Reliability of California Critical Thinking Disposition Inventory (CCTDI) in Kermanshah University of Medical Sciences. Educational Research in Medical Sciences, 1(1).
Handayani, R. & Wulandari, D. (2021). Modern Assessment dalam Menyongsong Pembelajaran Abad 21 dan Hambatan di Negara Berkembang. Jurnal Pendidikan Edutama, 8(1). https://doi.org/10.30734/jpe.v8i1.1363
Huang, L., Fan, A. P. C., Su, N., Thai, J., Kosik, R. O. & Zhao, X. (2021). Chinese medical students’ disposition for critical thinking: a mixed methods exploration. BMC Medical Education, 21(1). https://doi.org/10.1186/s12909-021-02801-w
Jing, S. (2021). Discussion on the Logical Basis of Critical Thinking. International Journal of Philosophy, 9(2). https://doi.org/10.11648/j.ijp.20210902.14
Li, W. & Guo, Q. (2021). Plotting receiver operating characteristic and precision–recall curves from presence and background data. Ecology and Evolution, 11(15). https://doi.org/10.1002/ece3.7826
Macalli, M., Navarro, M., Orri, M., Tournier, M., Thiébaut, R., Côté, S. M. & Tzourio, C. (2021). A machine learning approach for predicting suicidal thoughts and behaviours among college students. Scientific Reports, 11(1). https://doi.org/10.1038/s41598-021-90728-z
Magsino, L. (2021). Self-Regulation Learning Variables and Learners’ Performance: A Correlational Analysis. International Review of Social Sciences Research, 1(1). https://doi.org/10.53378/346498
Malhotra, S. & Karanicolas, J. (2020). A Numerical Transform of Random Forest Regressors corrects Systematically-Biased Predictions. ArXiv.
McCullough, C. (2019). Review of “agricultural involution: the processes of ecological change in Indonesia” by Clifford Geertz. International Journal of Anthropology and Ethnology, 3(1). https://doi.org/10.1186/s41257-019-0021-y
Mosimege, M. & Winnaar, L. (2021). Teachers’ instructional strategies and their impact on learner performance in grade 9 mathematics: Findings from timss 2015 in South Africa. Perspectives in Education, 39(2). https://doi.org/10.18820/2519593X/pie.v39.i2.22
Mu, Z. (2021). Application of Big Data Technology in the Cultivation Mode of Compound Talents in Civil Engineering. Lecture Notes on Data Engineering and Communications Technologies, 81. https://doi.org/10.1007/978-3-030-79197-1_19
Müller, N. M. & Seufert, T. (2018). Effects of self-regulation prompts in hypermedia learning on learning performance and self-efficacy. Learning and Instruction, 58. https://doi.org/10.1016/j.learninstruc.2018.04.011
Muschelli, J. (2020). ROC and AUC with a Binary Predictor: a Potentially Misleading Metric. Journal of Classification, 37(3). https://doi.org/10.1007/s00357-019-09345-1
Nguyen, T., Mengersen, K., Sous, D. & Liquet, B. (2023). SMOTE-CD: SMOTE for compositional data. PLoS ONE, 18(6 June). https://doi.org/10.1371/journal.pone.0287705
Nickl, M., Huber, S. A., Sommerhoff, D., Codreanu, E., Ufer, S. & Seidel, T. (2022). Video-based simulations in teacher education: the role of learner characteristics as capacities for positive learning experiences and high performance. International Journal of Educational Technology in Higher Education, 19(1). https://doi.org/10.1186/s41239-022-00351-9
Obuchowski, N. A. & Bullen, J. A. (2018). Receiver operating characteristic (ROC) curves: Review of methods with applications in diagnostic medicine. Physics in Medicine and Biology, 63(7). https://doi.org/10.1088/1361-6560/aab4b1
Ramírez-Montoya, M. S., Castillo-Martínez, I. M., Sanabria-Z, J. & Miranda, J. (2022). Complex Thinking in the Framework of Education 4.0 and Open Innovation—A Systematic Literature Review. Journal of Open Innovation: Technology, Market, and Complexity, 8(1). https://doi.org/10.3390/joitmc8010004
Rigatti, S. J. (2017). Random Forest. Journal of Insurance Medicine (New York, N.Y.), 47(1). https://doi.org/10.17849/insm-47-01-31-39.1
Sari, R. M., Sumarmi, Astina, I. K., Utomo, D. H. & Ridhwan. (2021). Increasing Students Critical Thinking Skills and Learning Motivation Using Inquiry Mind Map. International Journal of Emerging Technologies in Learning, 16(3). https://doi.org/10.3991/ijet.v16i03.16515
Sharma, H. L. & Priyamvada. (2022). Innovative Teaching Strategies To Foster Critical Thinking: a Review. International Journal of Creative Research Thoughts, 10(5).
Su, J., He, S. & Wu, Y. (2022). Features selection and prediction for IoT attacks. High-Confidence Computing, 2(2). https://doi.org/10.1016/j.hcc.2021.100047
Syed Marzuki, S. Z., Osman, C. A., Buyong, S. Z. & Mohammad Kamaruddin, L. (2020). Empirical Study on Motivation: Embedding design thinking mini project. Environment-Behaviour Proceedings Journal, 5(15). https://doi.org/10.21834/ebpj.v5i15.2458
Uwineza, I., Uworwabayeho, A. & Yokoyama, K. (2023). Effects of Interactive Mathematics Software on Grade-5 Learners’ Performance. International Journal of Learning, Teaching and Educational Research, 22(1). https://doi.org/10.26803/ijlter.22.1.10
Vergaray, A. D., Guerra, C., Cervera, N. & Burgos, E. (2022). Predicting Academic Performance using a Multiclassification Model: Case Study. International Journal of Advanced Computer Science and Applications, 13(9). https://doi.org/10.14569/IJACSA.2022.01309102
Vickers, A. J. & Elkin, E. B. (2006). Decision curve analysis: A novel method for evaluating prediction models. Medical Decision Making, 26(6). https://doi.org/10.1177/0272989X06295361
Yi, D., Wu, J., Zhang, M., Zeng, Q., Wang, J., Liang, J. & Cai, Y. (2022). Does Involution Cause Anxiety? An Empirical Study from Chinese Universities. International Journal of Environmental Research and Public Health, 19(16). https://doi.org/10.3390/ijerph19169826
Yi, F., Yang, H., Chen, D., Qin, Y., Han, H., Cui, J., Bai, W., Ma, Y., Zhang, R. & Yu, H. (2023). XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease. BMC Medical Informatics and Decision Making, 23(1). https://doi.org/10.1186/s12911-023-02238-9
Zeng, Q. (2023). Analysis of the Phenomenon of Chinese Educational Involution and Recommendations. BCP Business & Management, 41. https://doi.org/10.54691/bcpbm.v41i.4419
Zhang, C., Fan, H., Xia, J., Guo, H., Jiang, X. & Yan, Y. (2017). The Effects of Reflective Training on the Disposition of Critical Thinking for Nursing Students in China: A Controlled Trial. Asian Nursing Research, 11(3). https://doi.org/10.1016/j.anr.2017.07.002
ŽivkoviĿ, S. (2016). A Model of Critical Thinking as an Important Attribute for Success in the 21st Century. Procedia - Social and Behavioral Sciences, 232. https://doi.org/10.1016/j.sbspro.2016.10.034

No competing interests reported.

The influence of anti-involution training on the critical thinking of young healthcare professionals in dental outpatient clinics-based on machine learning model

Status:

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

Introduction

Materials and Methods

Participants

Data collection

statistical analysis

Analytical methods

Data preprocessing

Variable selection and prediction model establishment

Model performance evaluation

Interpretability of the model

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1