Patient Clinical Data
Comparison of patient clinical data between the group with lymph node metastasis and the group without metastasis (Table 1) showed notable variations in tumor size, grade, and lymphovascular invasion (LVI) (p<0.001), with no significant differences in other clinical characteristics observed between the groups. Additional multivariate analysis verified that the dimensions of the tumor [OR (95%CI) 1.191(1.164-1.219), p<0.001], the grade [OR (95%CI) 1.104(1.079-1.130), p<0.001], and the presence of LVI [OR (95%CI) 1.105(1.080-1.131), p<0.001] were autonomous factors that increased the risk of lymph node metastasis in cases of breast cancer. Table 2 indicates no significant variances in initial clinical features between the training and testing groups.
Table 1 Comparison of clinical characteristics between participants in the non-LNM and LNM groups
|
non-LNM
(N=464)
|
LNM (N=513)
|
Overall (N=977)
|
P-value
|
Age (years)
|
50.8±10.9
|
50.8±10.3
|
50.8±10.5
|
0.999
|
Position
|
|
|
|
0.997
|
Right
|
229 (49.4%)
|
252 (49.1%)
|
481 (49.2%)
|
|
Left
|
235 (50.6%)
|
261 (50.9%)
|
496 (50.8%)
|
|
Pregnant
|
|
|
|
0.982
|
No
|
32.0 (6.9%)
|
37.0 (7.2%)
|
69.0 (7.1%)
|
|
Yes
|
432 (93.1%)
|
476 (92.8%)
|
908 (92.9%)
|
|
Menopause
|
|
|
|
1
|
No
|
261 (56.3%)
|
289 (56.3%)
|
550 (56.3%)
|
|
Yes
|
203 (43.8%)
|
224 (43.7%)
|
427 (43.7%)
|
|
Nipple retraction
|
|
|
|
0.436
|
No
|
421 (90.7%)
|
477 (93.0%)
|
898 (91.9%)
|
|
Yes
|
43.0 (9.3%)
|
36.0 (7.0%)
|
79.0 (8.1%)
|
|
Nipple.discharge
|
|
|
|
0.46
|
No
|
448 (96.6%)
|
487 (94.9%)
|
935 (95.7%)
|
|
Yes
|
16.0 (3.4%)
|
26.0 (5.1%)
|
42.0 (4.3%)
|
|
Number.of.tumor
|
|
|
|
0.872
|
Single
|
436 (94.0%)
|
486 (94.7%)
|
922 (94.4%)
|
|
Multiple
|
28.0 (6.0%)
|
27.0 (5.3%)
|
55.0 (5.6%)
|
|
US tumor size (mm)
|
25.8±10.2
|
36.2±14.1
|
31.2±13.4
|
<0.001
|
Aspect ratio
|
|
|
|
0.508
|
≤1
|
410 (88.4%)
|
465 (90.6%)
|
875 (89.6%)
|
|
>1
|
54.0 (11.6%)
|
48.0 (9.4%)
|
102 (10.4%)
|
|
US tumor borderline
|
|
|
|
0.461
|
Clear
|
47.0 (10.1%)
|
65.0 (12.7%)
|
112 (11.5%)
|
|
Blurring
|
417 (89.9%)
|
448 (87.3%)
|
865 (88.5%)
|
|
US.tumor.form.
|
|
|
|
0.98
|
Rule
|
23.0 (5.0%)
|
24.0 (4.7%)
|
47.0 (4.8%)
|
|
Lrregular
|
441 (95.0%)
|
489 (95.3%)
|
930 (95.2%)
|
|
US tumor blood
|
|
|
|
0.894
|
No
|
77.0 (16.6%)
|
91.0 (17.7%)
|
168 (17.2%)
|
|
Yes
|
387 (83.4%)
|
422 (82.3%)
|
809 (82.8%)
|
|
US BI-RADS
|
|
|
|
0.905
|
3
|
13.0 (2.8%)
|
12.0 (2.3%)
|
25.0 (2.6%)
|
|
4
|
374 (80.6%)
|
410 (79.9%)
|
784 (80.2%)
|
|
5
|
63.0 (13.6%)
|
81.0 (15.8%)
|
144 (14.7%)
|
|
6
|
14.0 (3.0%)
|
10.0 (1.9%)
|
24.0 (2.5%)
|
|
Calcification
|
|
|
|
0.941
|
No
|
148 (31.9%)
|
169 (32.9%)
|
317 (32.4%)
|
|
Yes
|
316 (68.1%)
|
344 (67.1%)
|
660 (67.6%)
|
|
Echo
|
|
|
|
0.731
|
Low-echo
|
396 (85.3%)
|
448 (87.3%)
|
844 (86.4%)
|
|
Iso-echo
|
46.0 (9.9%)
|
38.0 (7.4%)
|
84.0 (8.6%)
|
|
High-echo
|
22.0 (4.7%)
|
27.0 (5.3%)
|
49.0 (5.0%)
|
|
Pathological type
|
|
|
|
0.974
|
Others
|
45.0 (9.7%)
|
52.0 (10.1%)
|
97.0 (9.9%)
|
|
Invasive ductal carcinoma
|
419 (90.3%)
|
461 (89.9%)
|
880 (90.1%)
|
|
Grade
|
|
|
|
<0.001
|
1
|
170 (36.6%)
|
110 (21.4%)
|
280 (28.7%)
|
|
2
|
227 (48.9%)
|
237 (46.2%)
|
464 (47.5%)
|
|
3
|
67.0 (14.4%)
|
166 (32.4%)
|
233 (23.8%)
|
|
LVI
|
|
|
|
<0.001
|
No
|
303 (65.3%)
|
205 (40.0%)
|
508 (52.0%)
|
|
Yes
|
161 (34.7%)
|
308 (60.0%)
|
469 (48.0%)
|
|
Ki67
|
|
|
|
0.932
|
No
|
119 (25.6%)
|
137 (26.7%)
|
256 (26.2%)
|
|
Yes
|
345 (74.4%)
|
376 (73.3%)
|
721 (73.8%)
|
|
CK7
|
|
|
|
0.534
|
No
|
243 (52.4%)
|
287 (55.9%)
|
530 (54.2%)
|
|
Yes
|
221 (47.6%)
|
226 (44.1%)
|
447 (45.8%)
|
|
EGFR
|
|
|
|
0.86
|
No
|
357 (76.9%)
|
387 (75.4%)
|
744 (76.2%)
|
|
Yes
|
107 (23.1%)
|
126 (24.6%)
|
233 (23.8%)
|
|
ER
|
|
|
|
0.902
|
No
|
242 (52.2%)
|
275 (53.6%)
|
517 (52.9%)
|
|
Yes
|
222 (47.8%)
|
238 (46.4%)
|
460 (47.1%)
|
|
HER2
|
|
|
|
0.729
|
No
|
176 (37.9%)
|
182 (35.5%)
|
358 (36.6%)
|
|
Yes
|
288 (62.1%)
|
331 (64.5%)
|
619 (63.4%)
|
|
PR
|
|
|
|
0.983
|
No
|
171 (36.9%)
|
192 (37.4%)
|
363 (37.2%)
|
|
Yes
|
293 (63.1%)
|
321 (62.6%)
|
614 (62.8%)
|
|
Table 2 Univariate and Multivariate analysis of risk factors related to LNM in Breast Cancer
Variable
|
Univariate analysis
|
Multivariate analysis
|
OR (95%CI)
|
P value
|
OR (95%CI)
|
P value
|
US tumor size
|
1.212(1.183-1.242)
|
0.00
|
1.191(1.164-1.219)
|
0.00
|
Grade
|
1.121(1.093-1.150)
|
0.00
|
1.104(1.079-1.130)
|
0.00
|
LVI
|
1.135(1.106-1.164)
|
0.00
|
1.105(1.080-1.131)
|
0.00
|
Performance of the Deep Learning Model
We trained Densenet121 as a deep feature extraction model for this analysis, extracting 50,176 deep features from each ROI using a pre-trained CNN. Furthermore, PCA was employed to decrease the dimension of pixel-level features to the top 512 most significant features (Supplementary File 1). Subsequently, with the lymph node status of breast cancer as the outcome target, Spearman and LASSO were applied to reduce the extracted deep features to 32 breast cancer lymph node metastasis-related features (Fig. 3A and 3B). Various machine learning algorithms were utilized to build models by integrating the filtered deep characteristics. The LR model showed the best performance in the training set based on the AUROC metric, as depicted in Fig. 3C and Supplementary Table 1. The LR, NaiveBayes, SVM, KNN, RandomForest, ExtraTrees, XGBoost, LightGBM, GradientBoosting, AdaBoost, and MLP models achieved accuracies of 0.772, 0.755, 0.745, 0.497, 0.517, 0.544, 0.684, 0.667, 0.639, 0.680, and 0.735, correspondingly, in the independent testing set (Fig. 3E). Table 4 and Fig. 3F display the AUC (95%CI) values of 0.823 (0.775-0.872), 0.790 (0.738-0.843), 0.796 (0.744-0.849), 0.590 (0.528-0.653), 0.630 (0.567-0.693), 0.603 (0.539-0.666), 0.719 (0.661-0.777), 0.710 (0.651-0.769), 0.683 (0.623-0.744), 0.684 (0.623-0.745), and 0.772 (0.718-0.826), demonstrating that the LR model performed the best in the testing dataset. The LR model in the testing set had a sensitivity of 0.835, specificity of 0.699, PPV of 0.763, and NPV of 0.785. This result demonstrates the high efficacy and stability of the LR model algorithm in predicting lymph node status based on deep features of breast cancer.
Table 3 Comparison of baseline characteristics between the training set and the test set
|
Training-set (N=683)
|
Test-set (N=294)
|
P-value
|
Age (years)
|
50.4±10.4)
|
51.9±10.8
|
0.114
|
Position
|
|
|
0.642
|
Right
|
343 (50.2%)
|
138 (46.9%)
|
|
Left
|
340 (49.8%)
|
156 (53.1%)
|
|
Pregnant
|
|
|
0.831
|
No
|
46.0 (6.7%)
|
23.0 (7.8%)
|
|
Yes
|
637 (93.3%)
|
271 (92.2%)
|
|
Menopause
|
|
|
0.0676
|
No
|
401 (58.7%)
|
149 (50.7%)
|
|
Yes
|
282 (41.3%)
|
145 (49.3%)
|
|
Nipple retraction
|
|
|
0.474
|
No
|
623 (91.2%)
|
275 (93.5%)
|
|
Yes
|
60.0 (8.8%)
|
19.0 (6.5%)
|
|
Nipple.discharge
|
|
|
0.663
|
No
|
651 (95.3%)
|
284 (96.6%)
|
|
Yes
|
32.0 (4.7%)
|
10.0 (3.4%)
|
|
Number.of.tumor
|
|
|
0.908
|
Single
|
646 (94.6%)
|
276 (93.9%)
|
|
Multiple
|
37.0 (5.4%)
|
18.0 (6.1%)
|
|
US tumor size (mm)
|
31.4±13.3
|
30.8±13.7
|
0.826
|
Aspect ratio
|
|
|
0.701
|
≤1
|
608 (89.0%)
|
267 (90.8%)
|
|
>1
|
75.0 (11.0%)
|
27.0 (9.2%)
|
|
US tumor borderline
|
|
|
0.881
|
Clear
|
76.0 (11.1%)
|
36.0 (12.2%)
|
|
Blurring
|
607 (88.9%)
|
258 (87.8%)
|
|
US.tumor.form.
|
|
|
0.245
|
Rule
|
38.0 (5.6%)
|
9.00 (3.1%)
|
|
Lrregular
|
645 (94.4%)
|
285 (96.9%)
|
|
US tumor blood
|
|
|
0.59
|
No
|
123 (18.0%)
|
45.0 (15.3%)
|
|
Yes
|
560 (82.0%)
|
249 (84.7%)
|
|
US BI-RADS
|
|
|
0.964
|
3
|
16.0 (2.3%)
|
9.00 (3.1%)
|
|
4
|
544 (79.6%)
|
240 (81.6%)
|
|
5
|
105 (15.4%)
|
39.0 (13.3%)
|
|
6
|
18.0 (2.6%)
|
6.00 (2.0%)
|
|
Calcification
|
|
|
0.302
|
No
|
232 (34.0%)
|
85.0 (28.9%)
|
|
Yes
|
451 (66.0%)
|
209 (71.1%)
|
|
Echo
|
|
|
0.834
|
Low-echo
|
586 (85.8%)
|
258 (87.8%)
|
|
Iso-echo
|
59.0 (8.6%)
|
25.0 (8.5%)
|
|
High-echo
|
38.0 (5.6%)
|
11.0 (3.7%)
|
|
Pathological type
|
|
|
0.982
|
Others
|
67.0 (9.8%)
|
30.0 (10.2%)
|
|
Invasive ductal carcinoma
|
616 (90.2%)
|
264 (89.8%)
|
|
Grade
|
|
|
1
|
1
|
196 (28.7%)
|
84.0 (28.6%)
|
|
2
|
323 (47.3%)
|
141 (48.0%)
|
|
3
|
164 (24.0%)
|
69.0 (23.5%)
|
|
LVI
|
|
|
0.609
|
No
|
348 (51.0%)
|
160 (54.4%)
|
|
Yes
|
335 (49.0%)
|
134 (45.6%)
|
|
Ki67
|
|
|
1
|
No
|
179 (26.2%)
|
77.0 (26.2%)
|
|
Yes
|
504 (73.8%)
|
217 (73.8%)
|
|
CK7
|
|
|
0.997
|
No
|
370 (54.2%)
|
160 (54.4%)
|
|
Yes
|
313 (45.8%)
|
134 (45.6%)
|
|
EGFR
|
|
|
0.0469
|
No
|
505 (73.9%)
|
239 (81.3%)
|
|
Yes
|
178 (26.1%)
|
55.0 (18.7%)
|
|
ER
|
|
|
0.98
|
No
|
360 (52.7%)
|
157 (53.4%)
|
|
Yes
|
323 (47.3%)
|
137 (46.6%)
|
|
HER2
|
|
|
0.826
|
No
|
246 (36.0%)
|
112 (38.1%)
|
|
Yes
|
437 (64.0%)
|
182 (61.9%)
|
|
PR
|
|
|
0.897
|
No
|
257 (37.6%)
|
106 (36.1%)
|
|
Yes
|
426 (62.4%)
|
188 (63.9%)
|
|
LNM
|
|
|
0.88
|
No
|
328 (48.0%)
|
136 (46.3%)
|
|
Yes
|
355 (52.0%)
|
158 (53.7%)
|
|
Table 4 Comparison of the performance of ultrasound imaging machine learning models in the test set
Model
|
Acc
|
AUC
|
95% CI
|
Sens
|
Spec
|
PPV
|
NPV
|
F1
|
LR
|
0.772
|
0.823
|
0.7747 - 0.8718
|
0.835
|
0.699
|
0.763
|
0.785
|
0.798
|
NaiveBayes
|
0.755
|
0.790
|
0.7381 - 0.8427
|
0.766
|
0.743
|
0.776
|
0.732
|
0.771
|
SVM
|
0.745
|
0.796
|
0.7435 - 0.8485
|
0.671
|
0.831
|
0.822
|
0.685
|
0.739
|
KNN
|
0.497
|
0.590
|
0.5282 - 0.6525
|
0.082
|
0.978
|
0.812
|
0.478
|
0.149
|
RandomForest
|
0.517
|
0.630
|
0.5670 - 0.6928
|
0.247
|
0.831
|
0.629
|
0.487
|
0.355
|
ExtraTrees
|
0.544
|
0.603
|
0.5388 - 0.6664
|
0.291
|
0.838
|
0.676
|
0.504
|
0.407
|
XGBoost
|
0.684
|
0.719
|
0.6611 - 0.7771
|
0.747
|
0.610
|
0.690
|
0.675
|
0.717
|
LightGBM
|
0.667
|
0.710
|
0.6511 - 0.7685
|
0.684
|
0.647
|
0.692
|
0.638
|
0.688
|
GradientBoosting
|
0.639
|
0.683
|
0.6225 - 0.7441
|
0.620
|
0.662
|
0.681
|
0.600
|
0.649
|
AdaBoost
|
0.680
|
0.684
|
0.6230 - 0.7452
|
0.873
|
0.456
|
0.651
|
0.756
|
0.746
|
MLP
|
0.735
|
0.772
|
0.7177 - 0.8261
|
0.778
|
0.684
|
0.741
|
0.727
|
0.759
|
Acc: Accuracy; AUC: Area Under the Curve; Sens: Sensitivity; Spec: Specificity; PPV: Positive Predictive Value; NPV: Negative Predictive Value; F1: F1 Score
Performance of the Fusion Model
The deep features of breast tumors were used to calculate the predicted probability of lymph node metastasis for each subject (US signature) using the LR machine learning algorithm. Further, the US signature was integrated with the three independent clinical risk factors for breast cancer lymph node metastasis using the LR machine learning algorithm to construct a fusion model, as visualized in Fig. 4. Feature fusion further improved the model's performance. As shown in Table 5, the accuracy in the training and testing sets was 0.796 and 0.820, respectively. The AUC (95%CI) values were 0.863 (0.837-0.890) and 0.885 (0.847-0.922), with the results visualized in Fig. 5A. The sensitivity was 0.744 and 0.741, while the specificity was 0.854 and 0.912. The PPV was 0.846 and 0.907, and the NPV was 0.755 and 0.752. In order to confirm the practical significance of the model, clinical assessments showed that utilizing the fusion model for forecasting lymph node metastasis in breast cancer patients with a probability between 0.05 and 0.95 could result in a positive outcome (Fig. 5B). The confusion matrix in Fig. 5C shows True Negatives (TN) of 0.82 and True Positives (TP) of 0.79, indicating that the fusion model has higher predictive accuracy for non-lymph node metastasis patients than for lymph node metastasis patients. Fig. 5D illustrates the fusion model's ability to accurately predict lymph nodes and classify the risk of lymph node metastasis in breast cancer. The findings indicate that the fusion model, which combines tumor-deep features and clinical characteristics, is highly effective in predicting the risk of lymph node metastasis in breast cancer.
Table 5 Performance of the fusion model in the training set and the test set
|
Acc
|
AUC
|
95% CI
|
Sens
|
Spec
|
PPV
|
NPV
|
F1
|
Training-set
|
0.796
|
0.863
|
0.8367 - 0.8903
|
0.744
|
0.854
|
0.846
|
0.755
|
0.792
|
Test-set
|
0.820
|
0.885
|
0.8465 - 0.9225
|
0.741
|
0.912
|
0.907
|
0.752
|
0.815
|