Patient data collection
We initially collected clinical information for a total of 1036 patients from the hospital's publicly available data set. 236 patients were excluded due to incomplete patient data and related data. Finally, 800 patients were selected and included in the study. The training set contained real liver fibrosis medical record samples in recent five years, among which positive samples accounted for 9% and negative samples accounted for 91%. In the sample, the ratio of male to female was 65:95, and the age distribution was mainly middle-aged and elderly people (Fig. 1). Each sample mainly contains the following characteristics (Table 1).
Table 1.1
Sample data characteristics
Name | Name | Name | Name |
Sex | White-Ball-Ratio | Prealbumin | Beta-Microglobulin |
Total-Bilirubin | Alanine-Aminotransferase | Cholinesterase | Cystatin-C |
Direct-Bilirubin | Aspartate-Aminotransferase | Alpha-Fucosidase | Glucose |
Indirect-Bilirubin | Cereal-Grass/Grain-c | Lactate-Dehydrogenase | Total-Cholesterol |
Globulin | Total-Bile-Acid | Uric-Acid | Low-Density-Cholesterol |
Total-protein | Alkaline-Phosphatase | Urea-Nitrogen | Triglycerides |
Albumin | Glutamyl-Transpeptidase | Creatinine | High-Density |
Alpha-Fetoprotein | Type-Ⅲ-Procollagen-PⅢNP | Ⅳ-Col | laminin |
Hyaluronic acid | RBC | Hemoglobin | WBC |
Neutrophils | Lymphocytes | Platelet | Prothrombin-Time |
INR | lgG | lgA | lgM |
C3 | C4 | QS1 | QS2 |
QS3 | QS4 | QS5 | |
Table 1.2
Physiological characteristic data
Name | Description |
QS1 | History of liver disease |
QS2 | Somatosensory pain level |
QS3 QS4 QS5 | History of alcoholism smoking history Genetic history |
Data quality assessment
In the existing training data set, each sample contains 51 dimensions of features, including 46 dimensions of biochemical features and 5 dimensions of physiological features. The results in the following table can be obtained by calculating Line value rate, listed value rate, value rate of key fields, proportion of organ data, retention ratio of cleaning column, retention ratio of cleaning bank validity rate of field type in data sample (Table 2). Quality represents the data quality conformity of the data set. After the above calculation, the quality conformity of the training data set adopted in this experiment is 95.98%, which > 90%. The experimental data set belongs to a wide range of real effective data set.
Table 2
Data set Quality Assessment Form
Index | Result |
Hν(Line value rate) | 94.32% |
Lν(Listed value rate) | 92% |
Gν(Value rate of key fields) | 100% |
Zν(Proportion of organ data) | 91.80% |
LQν(Retention ratio of cleaning column) | 100% |
HQν(Retention ratio of cleaning bank) | 100% |
Yν(Validity rate of field type in data sample) | 93.80% |
$$\text{Q}\text{u}\text{a}\text{l}\text{i}\text{t}\text{y}= \frac{1}{7}\left({\text{H}}_{\text{v}}+{\text{L}}_{\text{v}}+{\text{G}}_{\text{v}}+{\text{Z}}_{\text{v}}+{\text{L}}_{\text{Q}\text{v}}+{\text{H}}_{\text{Q}\text{v}}+{\text{Y}}_{\text{V}}\right)$$
Abnormal data detection
In this experiment, the 3-Sigma algorithm is used to detect abnormal data of all features of the data set, and the results of abnormal detection are shown in the Fig. 2. In the partial feature detection results below (Fig. 2A), the median line of the rectangle represents the median of the data, and the upper and lower sides of the rectangle correspond to the upper quartile Q3 and the lower quartile Q1 respectively. The distance IQR = Q3-Q1 was obtained from upper quartile Q3 and lower quartile Q1. Two whiskers extend from the box shape up and down. The top of the box is the observed maximum value Max = Q3 + 1.5IQR, and the bottom of the box is the observed minimum value Min = Q1-1.5IQR. The data beyond the observation point on the rectangle is the outlier data. More intuitively, TSNE algorithm is used to verify whether the results of the above 3-Sigma algorithm after anomaly detection are effective. As shown in the Fig. 2B, 3-Sigma algorithm can complete the detection of abnormal data of most samples, but there is still deviation for some sample data. In the three-dimensional coordinate system, some outliers are labeled as abnormal sample data by TSNE. According to the label of the corresponding marker, the abnormal sample data is identified and abnormal detection is completed.
Feature screening experiment
In this experiment, we first draw the feature box diagram, and then calculate the Spearman correlation coefficient between different features. This coefficient was visualized as a thermal map and made into a feature weight visualization map, which was used to analyze the influence of features and screen out the features with high correlation to the pre-detection and determination of liver fibrosis, so as to ensure the accuracy of subsequent models.
Figure 3A is the sample distribution map of alanine aminotransferase characteristics for liver fibrosis pre-detection drawn by this method. It can be concluded that alanine aminotransferase is mainly distributed between 0 and 100 in patients without hepatic fibrosis disease and is more dispersed in patients with hepatic fibrosis disease. It can be judged that the characteristics of alanine aminotransferase have a certain influence on the results of experimental classification. However, the characteristics of triglyceride had no significant effect on the results of experimental classification (Fig. 3B). All the features in the training data set were analyzed by feature engineering according to the above methods, and some features with significant influence on the liver fibrosis pre-detection experiment were finally screened out. For the training dataset that has completed data preprocessing, the Spearman correlation coefficient between different features in the dataset is calculated, and the graph shows the form of thermal map after the coefficient matrix is visualized. The darker the area, the higher the correlation between the corresponding abscissa feature and the corresponding ordinate feature. Correspondingly, if the color of the region is lighter, the correlation between the abscissa and ordinate of the corresponding region is lower. By this can be filtered on characteristics. After evaluation, the 40-dimensional features obtained by this feature screening are in line with the common pathological principles of medical liver fibrosis (Fig. 3C). Therefore, it is considered that the screening results of this feature are effective and retain the features that have influence on the pre-detection of liver fibrosis. Finally, the feature influence weight is calculated according to the correlation coefficient between the sample results and Spearman features, and the feature influence weight is visualized as shown in Fig. 3D below.
Model parameter tuning
The liver fibrosis pre-detection model based on CNN-former is mainly composed of CNN convolutional neural network module, multi-head attention mechanism module and feedforward neural network module. The Fig. 4A and 4B shows the ablation experimental function of CNN convolutional neural network module, which respectively represents the convergence comparison of the model before adding CNN one-dimensional convolution to the network after other module parameters are fixed and the model after adding CNN one-dimensional convolution. The above ablation experiments of CNN convolutional neural network module can verify that one-dimensional convolution at this layer can effectively promote the convergence speed of the model, reduce the computational complexity of the model, and improve the computational efficiency of the model. Figure 4C shows the feature attention visualization of the multi-head attention mechanism on the input features. The horizontal axis coordinates represent the attention weight assigned by the input features in the multi-head attention mechanism network under the multi-head attention mechanism. The darker the color, the higher the attention weight occupied by the corresponding feature, and the lighter the color, the lower the attention weight occupied by the corresponding feature. Blue focuses on the positive sample classification, red focuses on the negative sample classification. The vertical axis coordinates represent the attention between features captured by different heads in the input multi-head attention mechanism network. At the same time, by fixing other module parameters, the loss function curves of multi-head attention mechanism and self-attention mechanism network under the same training set are compared. It can be concluded that the loss function in the multi-head attention network converges faster than that in the self-attention mechanism network, and the loss error value of the loss function is smaller (Fig. 4D).
The model complexity and fitting ability of the network can be effectively improved by feedforward neural network. Table 3 is the comparison table of layer number ablation experiments of the feedforward neural network. Using the 3-layer fully connected neural network as the feedforward neural network has the best performance on the training set and the test set. Therefore, in this experiment, the feedforward neural network module based on CNN-former's liver fibrosis pre-detection model adopts three fully connected layers to compose the module.
Table 3
Comparison table of feedforward neural network layer ablation experiment
Number of network layers | Accuracy of training set | Test set accuracy |
1 | 89.7% | 85.4% |
2 | 90.3% | 88.7% |
3 | 96.8% | 93.1% |
4 | 95.7% | 92.2% |
5 | 91.6% | 89.8% |
Parameter tuning based on multi-CNN-Former model
Compared with the above CNN-former model, the liver fibrosis pre-detection model based on multi-CNN-Former adds two new modules: multi-task parameter sharing and multi-task loss function. In this ablation experiment, the comparison experiment between soft parameter sharing and hard parameter sharing (Table 4) was used to pre-check liver fibrosis and related complications and determine the parameter sharing method in the multi-CNN-former experiment. Among them, hard parameter sharing realizes parameter sharing by adding input data to a parameter sharing layer before entering the multi-task mechanism, and soft parameter sharing realizes parameter sharing by establishing the network layer of each sub-task. Comparing the model evaluation indexes under the sharing of hard and soft parameters, the following conclusions can be obtained. The accuracy and AUC values of the hard parameter sharing mechanism in this experiment are higher than those of the soft parameter sharing mechanism, and the hard parameter sharing mechanism has fewer parameters compared with the soft parameter sharing mechanism, which is more conducive to the model to complete the multi-task learning of liver fibrosis.
Table 4
Comparison table of parameter sharing ablation experiment
Evaluating indicator | Soft parameter sharing | Hard parameter sharing |
Precious | 87.81% | 88.18% |
Recall | 81.23% | 79.69% |
AUC | 82% | 83% |
F1 | 84.39% | 83.72% |
Model comparison experiment
The accuracy, recall, AUC value, F1 value and other evaluation indexes of the model were calculated based on the test set and confusion matrix to compare the effects of various models in predicting liver fibrosis pre-detection (Table 5). The CNN-former model proposed in this paper, without the help of imaging data, has a slightly lower accuracy, AUC value and recall rate in the test set than the imaging method. However, this method does not use imaging data, thus reducing the cost of pre-examination and improving the efficiency. Moreover, the actual pre-examination results still have certain reference value, which can be effectively carried out in the environment of lack of imaging medical equipment at the primary level.
Table 5
Results of method comparison experiment
Type | Data | Method | P | R | F1 | AUC |
Imaging | Based on CT image | Lubner | 0.790 | 0.848 | 0.790 | 0.860 |
Li | 0.830 | 0.833 | 0.830 | 0.857 |
Based on MRI image | House | 0.911 | 0.837 | 0.911 | 0.850 |
Based on ultrasonic image | YuH | 0.931 | 0.831 | 0.931 | 0.967 |
Baek | 0.922 | 0.851 | 0.922 | 0.868 |
Chan | 0.923 | 0.830 | 0.923 | 0.858 |
Based on image group | Wang | 0.957 | 0.835 | 0.957 | 0.980 |
Non-imaging | Based on four items of liver | GDBT | 0.851 | 0.807 | 0.851 | 0.879 |
RF | 0.854 | 0.833 | 0.854 | 0.884 |
Based on biochemical physiology | CNN-Former | 0.904 | 0.821 | 0.904 | 0.910 |
Multi-CNN-Former | 0.891 | 0.861 | 0.891 | 0.880 |