1.Ultrasound images
This retrospective study was approved by the Ethics Committee of Anhui Provincial Children's Hospital(approval no. 20190021), and all methods were carried out in accordance with their guidelines and regulations.The requirement for informed consent was waived because the Ethics Committee of Anhui Provincial Children's Hospital used completely anonymised data.2021 standardized ultrasound images (Philips Color Ultrasonic Diagnostic System EPIQ5, 7–12 Hz) were recorded in the outpatient department of hospital from January 2019 to January 2021(Fig. 1). The recorded ultrasonic image segments are taken from the beginning of operation to the end of ultrasonic diagnosis of the operator, and the image used for diagnosis is saved at the same time.The hip joint was placed in a physiologically neutral position due to the influence of different positions on the β angle [16]. The ultrasonic probe was placed on coronal plane and rotated backward about 10–15 degrees.2021ultrasound image fragments were finally generated; the process is shown in Fig. 1.
2.Obtaining cross-sections of ultrasound images
The Graf screening method can only be performed with the cross-section containing standard ultrasound images.The specific screening process is shown in Fig. 1. Then it was submitted to the labeling team to complete the diagnostic image labeling.
3.The annotation group completed the true value of the image annotation
Six key points were labeled on standard hip joint ultrasound images according to Graf theory.The baseline, bone parietal line, and cartilage apical lines were plotted (Fig. 2). The rectus femoris tendon was used as the starting point, and the iliac crest border was connected as the baseline.Connect the lowest point of the ilium with the osseous apex of the acetabulum as the bone parietal line. The drawing method of cartilage apical line is to connect the turning point of cartilage apical from concave to convex with the center of labrum. Graf classification was then determined using the α and β angles (α, between bone parietal line and baseline; β, between cartilage apical line and baseline). 268 standard hip ultrasound images were randomly selected as the test set and labeled.The annotation process is also shown in Fig. 1. Finally the data in the test set could be divided into six groups according to Graf classification and two groups according to joint maturity (Table 1).
Table 1
Characteristic | Entire set (n = 2021) | Training set (n = 1753) | Testing set (n = 268) |
Sex | | | |
Female | 1700 | 1476 | 224 |
Male | 321 | 277 | 44 |
Age | | | |
0–60 days | 795 | 682 | 113 |
≥ 61–120 days | 880 | 776 | 104 |
≥ 121–180 months | 346 | 295 | 51 |
Graf | | | |
Non-dislocation | 1779 | 1552 | 227 |
IA | 1442 | 1241 | 201 |
IB | 335 | 311 | 24 |
Dislocation | 242 | 201 | 41 |
II | 184 | 149 | 35 |
Stable IIC | 32 | 29 | 3 |
Unstable IIC, D | 3 | 3 | 0 |
IIIA/B, IV | 25 | 20 | 5 |
-
Network framework.
As shown in the Fig. 3, we applied a deep learning method for the automatic diagnosis of ultrasound images. The localization of a landmark was converted to the regression of a heat map, centered at landmark. For an input image, our network first utilized a deep-learning baseline, named ResNet-50, to obtain a rich feature representation of spatial information. Based on the feature map, we used deconvolutional layers to decode the feature map and generate target heat map which indicate the position of the landmark. Finally, the system completed the diagnosis with the position of landmarks.
Heat map regression network use mean square error (MSE) loss to optimize the network. To train the heat map regression network, we generated the target heat map according to the manual labeled landmark. The position of landmark has the maximum pixel value. The pixel value of other positions depend on the distance to the landmark:
$$\text{H}\left(\text{x},\text{y}\right)=-\text{exp}\left\{\frac{{\left(x-{x}^{\ast }\right)}^{2}+{\left(y-{y}^{\ast }\right)}^{2}}{2{\sigma }^{2}}\right\},$$
where \({x}^{\ast },{y}^{\ast }\)refer to the coordinate of the landmark, \(\text{H}\left(\text{x},\text{y}\right)\) represents the target heat map. \(\sigma\) is a hyper-parameter which represent the variance, we use \(\sigma =3\) pixels in our work.
After the training of heat map regression network, our method can learn the potential position of landmark by generating the heat map. The landmark can be determined according to the heat map by selecting the position with maximum pixel value. Finally, the α and β angles and illness analysis can be determined by the hip’s morphology according to the landmarks.
5. System test
The authors conducted receiver operating characteristic (ROC) tests on the consistency of the relevant parameters between the AI system and the clinicians on the test set. Subsequently, the ROC curves of the two groups were compared according to clinical diagnosis of maturity (the “mature” group included IA and IB, “immature” included IIA/B, stable IIC, unstable IIC, D, IIIA/B, and IV). The α and β angles measured by the AI system were compared with those measured by clinicians.
6. Statistical analyses
Data were analyzed using SPSS 22.0 (IBM Corp., Armonk, NY, USA) and GraphPad Prism 5 (GraphPad Inc., San Diego, CA, USA). ROC curves were used to evaluate the diagnostic performance of the AI system in determining hip joints maturity. Bland–Altman scatter plots were then used to evaluate consistency between the AI system and the clinician-measured acetabular index. When P < 0.05, the difference was statistically significant.