Accuracy of Artificial Intelligence-Assisted Landmark Identification in Serial Lateral Cephalograms of Class III Patients Who Underwent Two-Jaw Orthognathic Surgery

doi:10.21203/rs.3.rs-687966/v1

To compare the accuracy of artificial intelligence-assisted landmark identification in serial lateral cephalograms of Class III patients who underwent two-jaw orthognathic surgery using a convolutional neural network (CNN) algorithm. 3,188 lateral cephalograms of Class III patients were allocated into the training and validation sets (3,004 cephalograms of 751 patients) and test set (184 cephalograms of 46 patients; subdivided into the genioplasty and non-genioplasty groups, n=23 per group)]. Each patient in the test set had four cephalograms: initial (T0), pre-surgery [T1, presence of orthodontic brackets (OBs)], post-surgery [T2, presence of OBs and surgical plates and screws (S-PS)], and debonding [T3, presence of S-PS and fixed retainers (FR)]. Statistical analysis was performed using mean errors of 20 landmarks between human gold standard and the CNN model. The total mean error was 1.17 mm without significant difference among four time-points. Before and after surgery, ANS, A point, and B point showed an increased error, while Mx6D and Md6D showed a decreased error. No difference in errors existed at B point, Pogonion, Menton, Md1C, and Md1R between the genioplasty and non-genioplasty groups. The CNN model can be used for landmark identification in serial cephalograms despite presence of OB, S-PS, FR, genioplasty, and bone remodeling.

Health Policy

Biomedical Engineering

artificial intelligence

cephalograms

convolutional neural network (CNN)

orthodontic brackets

Owing to the high prevalence of Class III malocclusion and negative social recognition of the prognathic appearance,^1,2 Korea has become one of the countries that performs two-jaw orthognathic surgery (TJ-OGS) extensively in patients with skeletal Class III malocclusion.To obtain successful treatment outcome, the following four steps should be performed precisely: (1) diagnosis and gross treatment planning for pre-surgical orthodontic treatment and orthognathic surgery using initial cephalograms, (2) planning for the direction and amount of surgical movement using pre-surgical cephalograms, (3) assessment of surgical outcome and planning for post-surgical orthodontic treatment using post-surgical cephalograms, and (4) comprehensive assessment of orthodontic treatment and orthognathic surgery using debonding cephalograms.^3,4 In addition, superimposition of serial cephalograms taken at different time-points is also important to assess the outcomes of pre- and post-surgical orthodontic treatment and orthognathic surgery. Accurate detection of cephalometric landmarks is mandatory to perform these procedures.

An artificial intelligence (AI) algorithm including convolutional neural network (CNN) can help clinicians detect cephalometric landmarks, whose accuracy is close to that of human experts.^5-12 Previous AI studies have regarded the accuracy within a range of 2 mm as a clinically acceptable performance in landmark identification.^8,12-15 However, it appears to be a lenient standard for appropriate clinical use. Therefore, use of stricter criteria (i.e., range within at least 1.5 mm) is necessary in determining the accuracy of landmark identification for clinical relevance.

In addition, most AI studies on the accuracy of automated landmark identification^8,13-15have trained and tested their models using initial lateral cephalograms only, which do not have orthodontic brackets (OB), surgical plates and screws (S-PS), fixed retainer (FR), and bone remodeling changes. To the best of our knowledge, no study has compared the accuracy of automated landmark identification in serial cephalograms at the four time-points covering from the initial, pre-surgery, post-surgery, to debonding stages in orthognathic surgery cases. Therefore, the purpose of the study was to compare the accuracy of AI-assisted landmark identification in serial lateral cephalograms of Class III patients who underwent pre- and post-surgical orthodontic treatment and TJ-OGS using a cascade CNN algorithm and strict criteria for determining the degree of accuracy.

Evaluation of total landmarks (Table 1)

The total landmarks showed a good mean error value (1.17 mm), and the total AP had a high degree of accuracy (74.2%) in Table 1.

Evaluation of skeletal landmarks (Table 1)

Nasion and Sella showed an excellent mean error value and a very high degree of accuracy (0.59 mm and 95.1%; 0.46 mm and 100%, respectively). Porion and Orbitale showed a good mean error value and a high degree of accuracy (1.07 mm and 76.1%; 1.21 mm and 73.9%, respectively). However, Basion showed a fair mean error value (1.64 mm) and a medium degree of accuracy (63.1%).

ANS and A point showed a good mean error value and a medium degree of accuracy (1.39 mm and 65.2%; 1.41 mm and 63.0%, respectively). PNS had a good mean error value (1.19 mm) and a high degree of accuracy (72.7%).

Pogonion, Menton and Articulare showed an excellent mean error value and a very high degree of accuracy (0.79 mm and 91.3%, 0.77 mm and 93.5%, 0.77 mm and 93.5%, respectively). B point showed a good mean error value (1.15 mm) and a high degree of accuracy (77.2 %).

Evaluation of dental landmarks (Table 1)

Mx1C showed an excellent mean error value (0.44 mm) and a very high degree of accuracy (97.8%). Mx6D had a good mean error value (1.43 mm) and a medium degree of accuracy (64.1%). However, Mx1R and Mx6R had a fair mean error value and a low degree of accuracy (1.55 mm and 57.6%; 1.68 mm and 51.6%, respectively).

Md1C demonstrated an excellent mean error value (0.49 mm) and a very high degree of accuracy (97.3%). Md1R had a fair mean error value (1.57 mm) and a low degree of accuracy (58.2%). Md6D had a fair mean error value (1.67 mm) and low accuracy (51.6%). Md6R exhibited an acceptable mean error value (2.03 mm) and a low degree of accuracy (41.3%).

Comparison of the mean errors among the four timepoints (T0, T1, T2, and T3) (Table 2)

No significant difference was found in the overall mean errors (P > 0.05). Only three landmarks including ANS, Mx6D, and Md6R showed a significant difference in the mean errors among the four timepoints [ANS, increase in the mean error from T0 and T1 to T2, P < 0.01; Mx6D, decrease in the mean error from T0 to T2, P < 0.05; Md6R, decrease in the mean error from T0 to T2 and T3, P < 0.01].

Comparison of the mean errors between the two timepoints [(T0, T1) vs. (T2, T3)] (Table 2)

ANS, A point, and B point showed an increase of mean error after TJ-OGS than before TJ-OGS, [ANS, P < 0.01; A point, P < 0.05; B point, P < 0.01], while Mx6D and Md6D showed a decrease in the mean error after TJ-OGS than before TJ-OGS [all P < 0.01].

Comparison of the mean errors between the genioplasty and non-genioplasty groups (Table 3)

No significant difference in the mean errors in the landmarks located adjacent to the genioplasty area (B point, Pogonion, Menton, Md1C, and Md1R) existed in each timepoint between the two groups, except Md1R at T1 (P<0.05).

Since TJ-OGS induces the position change and bone remodeling in the skeletal structures and produces the metallic images of the OB, SP-S, and FR, the accuracy and reliability of cephalometric landmark identification in serial lateral cephalograms are important for assessment of treatment outcomes.¹⁶ As total landmarks exhibited a good mean error value and a high degree of accuracy (1.17 mm and 74.2%, respectively, Table 1) without significant difference among the four time-points (P > 0.05, Table 2), accuracy of the AI-assisted digitization was not significantly affected by the presence of OB, SP-S, FR, and bone remodeling change during orthodontic treatment and TJ-OGS. Regardless of the degree of accuracy of each landmark (Table 1), none of the five cranial base landmarks exhibited a significant difference in the mean errors among the four time-points (T0, T1, T2 and T3) and between the two time-points [(T0, T1) vs. (T2, T3)] (Table 2). Accuracy of the cranial base landmarks can be regarded as baseline for comparison of serial lateral cephalograms because the positions of these cranial base landmarks are not affected by TJ-OGS.

Three error patterns were found in the maxillary skeletal landmarks. First, the mean errors of ANS were different among the four time-points (T0, 1.07 mm; T1, 1.22 mm; T2, 1.78 mm; T3, 1.49 mm, P < 0.01; Table 2) and presented an increased error value after TJ-OGS than before TJ-OGS [(T0, T1) vs. (T2, T3), P < 0.01; Table 2], which suggested that the metal image of the SP-S adjacent to ANS as well as surgical shape modification of ANS ^17,18 (Fig. 1) could affect the accuracy of AI-assisted landmark detection. Second, although the error of A point was not significantly different among the four time-points (T0, 1.27 mm; T1, 1.28 mm, T2, 1.50 mm, T3, 1.59 mm, Table 2), it presented an increase in the mean error value after TJ-OGS than before TJ-OGS [(T0, T1) vs. (T2, T3), P < 0.05; Table 2]. This occurred because A point might be less affected by the metal image of the SP-S installed at the maxilla and have a lower chance for surgical shape modification, compared to ANS (Fig. 1). Third, in case of posterior impaction and/or anteroposterior movement of the maxilla, the position of PNS had to be changed. However, for PNS, no significant difference was found either among the four time-points (T0, 1.16 mm; T1, 1.14 mm, T2, 1.29 mm, T3, 1.17 mm; P > 0.05, Table 2) or between the two time-points [(T0, T1) vs. (T2, T3), P > 0.05; Table 2]. This might be due to (1) absence of the metal image of the SP-S within the ROI of PNS and (2) the end point of the hard palate can still be easily defined.

There are three explanations of the errors in the mandibular skeletal landmarks. First, since there were no metal images within the ROI of Articulare and Menton, their mean errors were not significantly different among the four time-points and between the two time-points (all P > 0.05, Table 2). Second, the mean error of Pogonion was not significantly different among the four time-points and between the two time-points (P > 0.05; Table 2), which suggests that the metal image of the SP-S adjacent to Pognion (Fig. 1) might not affect the accuracy of AI-assisted landmark detection. Third, although the mean errors of B point did not differ among the four time-points (T0, 1.00 mm; T1, 1.01 mm; T2, 1.29 mm; T3, 1.31 mm, P > 0.05; Table 2), comparison of the two time-points revealed an increase in error after TJ-OGS than before TJ-OGS [(T0, T1) vs. (T2, T3), P < 0.01; Table 2]. These findings suggest that the metal image of the SP-S adjacent to the B point (Fig. 1) might affect the accuracy of AI-assisted landmark detection.

There are two sources of errors in the dental landmarks. First, regardless of the degree of accuracy in the dental landmarks (Table 1), Mx1C, Md1C, Mx1R, Md1R, Mx6R, and Md6R did not exhibit significant difference in the mean errors among the four time-points and between the two time-points (all P > 0.05; Table 2). Second, the mean errors of Mx6D and Md6D were significantly different among the four time-points (Mx6D: T0, 1.66 mm; T1, 1.63 mm, T2, 1.20 mm, T3, 1.23 mm; Md6D, T0, 2.15 mm; T1, 1.71 mm, T2, 1.51 mm, T3, 1.33 mm; all P < 0.01, Table 2) and presented decreased mean error values after TJ-OGS than before TJ-OGS [(T0, T1) vs. (T2, T3), all P < 0.01; Table 2]. Possible reasons might be as follows: (1) Horizontal and vertical overlapping of the right and left maxillary and mandibular first molars made it difficult to accurately locate the Mx6D and Mn6D at T0 lateral cephalogram; and (2) Orthodontic treatment and TJ-OGS improved the alignment of the maxillary and mandibular dentition and corrected the cant, shift and yaw of the maxilla and mandible, reducing the double images of the maxillary and mandibular first molars.

No significant difference was found in the mean errors in the landmarks adjacent to the genioplasty area including B point, Pogonion, Menton, Md1C, and Md1R (all P > 0.05, Table 3). Possible reasons might be as follows: (1) Menton and Md1C were located relatively far from the SP-S installed at the symphysis and their shapes were not affected by orthognathic surgery; (2) Since Pogonion and B point are the most forward and deepest points on the anterior surface of the symphysis, respectively, they can be easily identified despite the presence of the metal image of the SP-S; and (3) Although Md1R had a fair mean error value and a low degree of accuracy (1.57 mm and 58.2%, respectively), these patterns were not aggravated at T2 and T3 despite the presence of the metal image of the SP-S.

Although this study might provide meaningful results about the accuracy of AI-assisted landmark identification of the hard tissue landmarks in serial lateral cephalograms, further studies are needed to investigate the accuracy of soft tissue landmark identification in serial lateral cephalograms.

The cascade CNN algorithm proposed in this study can be used for landmark identification in serial lateral cephalograms despite the presence of OB, S-PS, FR, genioplasty, and bone remodeling.

Materials. A total of 3,188 lateral cephalograms of 797 patients with Class III malocclusion were used for the training and validation sets and the test set for automated landmark identification using the CNN model. All procedures were performed in accordance with relevant guidelines. The inclusion criteria were as follows: (1) Class III patient who underwent pre- and post-surgical orthodontic treatment and TJ-OGS with/without genioplasty and (2) Class III patient whose serial lateral cephalograms were available. The exclusion criterion was Class III patient who had craniofacial deformities.The training and validation sets for automated landmark identification by the CNN model included 3,004 lateral cephalograms of 751 Class III patients from 10 institutions (Table 4). Some of the patients who belonged to the training or validation set had more than four lateral cephalograms because additional progress lateral cephalograms were taken between time-points, while some of them had missing lateral cephalograms at a specific timepoint.

For the test set, Class III patients with cephalograms obtained at the following four timepoints were selected: initial (T0), pre-surgery (T1, taken at least 1 month before TJ-OGS; presence of OBs), post-surgery (T2, taken at least 2 months after TJ-OGS; presence of OBs and S-PS), and debonding [T3, presence of S-PS, FR, and bone remodeling change). As a result, the test set consisted of 184 cephalograms of 46 Class III patients from eight institutions (Table 4). It was subdivided into the genioplasty and non-genioplasty groups (n = 23 patients per group). Their characteristics are enumerated in Figure 1. The written documentation of informed consent was waived and approved by the decision of the Institutional Review Board (IRB) Committee of 10 institutions: Seoul National University Dental Hospital (ERI18002), Kyung Hee University Dental Hospital (KH-DT19006), Kyungpook National University Dental Hospital (KNUDH-2019-03-02-00), Asan Medical Center (2019-0408), Ewha University Medical Center (EUMC 2019-04-017-009), Wonkwang University Dental Hospital (WKDIRB201903-01), Ajou University Dental Hospital (AJIRB-MED-MDB-19-039), Korea University Anam Hospital (K2019-0543-010), Chonnam National University Dental Hospital (CNUDH-EXP-2021-001), and Chosun University Dental Hospital (CUDHIRB 1901 005 R01).

Data sets were obtained from 10 centers using anonymized Digital Imaging and Communications in Medicine (DICOM) file format. Since finding the exact location of landmarks in a large lateral cephalogram image is relatively difficult, a fully automated landmark prediction algorithm with the cascade network was developed.¹² Two steps were followed: 1) detection of the region of interest (ROI; 256 × 256 and 512 × 512 pixels depending on the landmark) using the RetinaNet¹⁹ and 2) prediction of the landmark using the U-Net²⁰ (Figure 2).

Definitions of 12 skeletal and eight dental landmarks are presented in Figure 3 and Table 5. The landmarks were digitized by a single orthodontist who had 20 years of experience (human gold standard, MHH) and by the CNN model. The mean values of absolute errors for each landmark were calculated using the absolute distance between the human gold standard and AI-assisted detection. The degree of error was allocated into excellent (< 1.0 mm), good (1.0 -1.5 mm), fair (1.5 - 2.0 mm), acceptable (2.0 - 2.5 mm), and unacceptable (> 2.5 mm) groups. Then, the accuracy percentage (AP) was calculated using a formula (percentage of the excellent and good groups among the total degree of error groups), which means that the error range within 1.5 mm was considered accurate. The degree of accuracy was defined as “very high” (AP > 90%), ”high” (AP, 70 ~ 90%), ”medium” (AP, 50 ~ 70%), and ”low” (AP < 50%). Repeated measures analysis of variance (ANOVA) test with Tukey HSD, repeated measures multivariate analysis of variance (MANOVA) test, and independent t-test were performed using SPSS ver. 23.0 (IBM Corp., Armonk, NY, USA). P-values of < 0.05 were considered statistically significant.

Acknowledgements: This research was supported by grants from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute and funded by the Ministry of Health &Welfare (HI18C1638) and the Technology Innovation Program (20006105) funded by the Ministry of Trade, Industry & Energy, Republic of Korea

Author Contributions: “Conceptualization, S.-H. B. and M.H.; methodology, I.K., N.K., and S.-H.B.; software, I.K. and N.K.; validation, M.H., I.K., N.K., and S.-H.B..; formal analysis, M.H. ; investigation, N.K., and S.-H.B.; resources, M.H., J. -H. C., K.-H.K., M.K., S.-J.K.,Y.-J.K., S.J.S., Y.H.K., S.-H.L. and S.-H.B.; data curation, I.K. and N.K.; writing—original draft preparation, M.H; writing—review and editing, I.K., J.-H.C., K.-H.K., M.K., S.-J. K., Y.-J.K., S.J.S., Y.H.K., S.-H.L., N.K. and S.-H.B.; supervision, N.K. and S.-H.B.; project administration, N.K., S.-H. B. and M.H.; funding acquisition, S.-H.B. and M.H. All authors have read and agreed to the published version of the manuscript."

Im DH, Kim TW, Nahm DS, Chang YI. Current trends in orthodontic patients in Seoul National University Dental Hospital. Korean J Orthod. 2003;33:63-72.
Piao Y, Kim SJ, Yu HS, Cha JY, Baik HS. Five-year investigation of a large orthodontic patient population at a dental hospital in South Korea. Korean J Orthod. 2016;46:137-45.
Nielsen IL. Maxillary superimposition: a comparison of three methods for cephalometric evaluation of growth and treatment change. Am J Orthod Dentofacial Orthop. 1989;95:422-31.
Johnston C, Burden D, Kennedy D, Harradine N, Stevenson M. Class III surgical-orthodontic treatment: a cephalometric study. Am J Orthod Dentofacial Orthop. 2006;130:300-9.
Hutton TJ, Cunningham S, Hammond P. An evaluation of active shape models for the automatic identification of cephalometric landmarks. Eur J Orthod. 2000;22:499-508.
Leonardi R, Giordano D, Maiorana F, Spampinato C. Automatic cephalometric analysis. Angle Orthod. 2008;78:145-51.
Leonardi R, Giordano D, Maiorana F. An evaluation of cellular neural networks for the automatic identification of cephalometric landmarks on digital images. J Biomed Biotechnol. 2009;2009:717102.
Arık SÖ, Ibragimov B, Xing L. Fully automated quantitative cephalometry using convolutional neural networks. J Med Imaging (Bellingham). 2017;4:014501.
Lee JH, Yu HJ, Kim MJ, Kim JW, Choi J. Automated cephalometric landmark detection with confidence regions using Bayesian convolutional neural networks. BMC Oral Health. 2020;20:270.
Vandaele R, Aceto J, Muller M, Péronnet F, Debat V, Wang CW, et al. Landmark detection in 2D bioimages for geometric morphometrics: a multi-resolution tree-based approach. Sci Rep. 2018;8:538.
Khanagar SB, Al-Ehaideb A, Vishwanathaiah S, Maganur PC, Patil S, Naik S, et al. Scope and performance of artificial intelligence technology in orthodontic diagnosis, treatment planning, and clinical decision-making - A systematic review. J Dent Sci. 2021;16:482-492.
Kim J, Kim I, Cho JH, Hong MH, Kang KH, Kim MJ, et al.Accuracy of automated identification of lateral cephalometric landmarks using cascade convolutional neural networks on lateral cephalograms from nationwide multi-centers. Orthod Craniofac Res. 2021 May 11. doi:10.1111/ocr.12493. Epub ahead of print
Wang CW, Huang CT, Hsieh MC, Li CH, Chang SW, Li WC, et al. Evaluation and Comparison of Anatomical Landmark Detection Methods for Cephalometric X-Ray Images: A Grand Challenge. IEEE Trans Med Imaging. 2015;34:1890-900.
Wang CW, Huang CT, Lee JH, Li CH, Chang SW, Siao MJ, et al. A benchmark for comparison of dental radiography analysis algorithms. Med Image Anal. 2016;31:63-76.
Hwang HW, Park JH, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: Part 2-Might it be better than human?. Angle Orthod. 2020;90:69-76.
Roden-Johnson D, English J, Gallerano R. Comparison of hand-traced and computerized cephalograms: landmark identification, measurement, and superimposition accuracy. Am J Orthod Dentofacial Orthop. 2008;133:556-64.
Seigo O, Noriko N, Yuya N, Hitoshi Y, Tokutano M, Takako K, et al. Effects of vertical movement of the Anterior Nasal Spine on the Maxillary stability after LeFort 1 Osteotomy for Pitch correction. J Craniofacial Surg. 2015;26:e481-485
Venkategowda PR, Prakash AT, Roy ET, Shetty KS, Thakkar S, Maurya R. Stability of vertical, horizontal and angular parameters following superior repositioning of Maxilla by Le Fort I osteotomy: A cephalometric study. J Clin Diagn Res. 2017;11:ZC10-ZC14.
Lin TY, Goyal P, Girshick R, He K, Dollar P. Focal Loss for Dense Object Detection. IEEE Trans Pattern Anal Mach Intell. 2020;42:318-327.
Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597v1

Table 1. The absolute values of error, distribution of error, accuracy percentage, and degree of accuracy for each landmark

Compartment			Landmark	Absolute value of error		Distribution (number/percentage)					Accuracy
Compartment			Landmark	Mean (mm)	SD (mm)	Excellent (< 1.0 mm)	Good (1.0 -1.5 mm)	Fair (1.5-2.0 mm)	Acceptable (2.0-2.5 mm)	Unacceptable (> 2.5 mm)	Accuracy Percentage (number/percentage)	Degree of Accuracy
Skeletal landmark	Cranial Base		Nasion	0.59	0.48	157 (85.3%)	18 (9.8%)	4 (2.2%)	3 (1.6%)	2 (1.1%)	175 (95.1%)	Very high
			Sella	0.46	0.23	180 (97.8%)	4 (2.2%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	184 (100%)	Very high
			Porion	1.07	0.69	103 (56.0%)	37 (20.1%)	24 (13.0%)	14 (7.6%)	6 (3.3%)	140 (76.1%)	high
			Orbitale	1.21	1.01	92 (50.0%)	44 (23.9%)	21 (11.4%)	12 (6.5%)	15 (8.2%)	136 (73.9%)	high
			Basion	1.64	1.61	82 (44.6%)	34 (18.5%)	21 (11.4%)	13 (7.1%)	34 (18.5%)	116 (63.1%)	medium
	Maxilla	anterior	ANS	1.39	1.01	72 (39.1%)	48 (26.1%)	23 (12.5%)	14 (7.6%)	27 (14.7%)	120 (65.2%)	medium
		anterior	A point	1.41	0.99	86 (46.7%)	30 (16.3%)	22 (12.0%)	18 (9.8%)	28 (15.2%)	116 (63.0%)	medium
		posterior	PNS	1.19	0.89	97 (52.7%)	37 (20.1%)	24 (13.0%)	13 (7.1%)	13 (7.1%)	134 (72.7%)	high
	Mandible	anterior	B point	1.15	0.96	106 (57.6%)	36 (19.6%)	21 (11.4%)	8 (4.3%)	13 (7.1%)	142 (77.2%)	high
		anterior	Pogonion	0.79	0.68	140 (76.1%)	28 (15.2%)	7(3.8%)	1 (0.5%)	8 (4.3%)	168 (91.3%)	Very high
		bottom	Menton	0.77	0.44	143 (77.7%)	29 (15.8%)	8 (4.3%)	3 (1.6%)	1 (0.5%)	171 (93.5%)	Very high
		posterior	Articulare	0.77	0.45	138 (75.0%)	34 (18.5%)	10 (5.4%)	1 (0.5%)	1 (0.5%)	172 (93.5%)	Very high
Dental landmark	Maxillary dentition	anterior	Mx1C	0.44	0.37	178 (96.7%)	2 (1.1%)	2 (1.1%)	1 (0.5%)	1 (0.5%)	180 (97.8%)	Very high
		anterior	Mx1R	1.55	1.09	63 (34.2%)	43 (23.4%)	36 (19.6%)	12 (6.5%)	30 (16.3%)	106 (57.6%)	low
		posterior	Mx6D	1.43	1.08	76 (41.3%)	42 (22.8%)	23 (12.5%)	17 (9.2%)	26 (14.1%)	118 (64.1%)	medium
		posterior	Mx6R	1.68	1.08	51 (27.7%)	44 (23.9%)	35 (19.0%)	19 (10.3%)	35 (19.0%)	95 (51.6%)	low
	Mandibular dentition	anterior	Md1C	0.49	0.64	172 (93.5%)	7 (3.8%)	0 (0.0%)	2 (1.1%)	3 (1.6%)	179 (97.3%)	Very High
		anterior	Md1R	1.57	1.04	64 (34.8%)	43 (23.4%)	29 (15.8%)	18 (9.8%)	30 (16.3%)	107 (58.2%)	low
		posterior	Md6D	1.67	1.24	54 (29.3%)	41 (22.3%)	33 (17.9%)	30 (16.3%)	26 (14.1%)	95 (51.6%)	low
		posterior	Md6R	2.03	1.35	46 (25.0%)	30 (16.3%)	25 (13.6%)	28 (15.2%)	55 (29.9%)	76 (41.3%)	low
Total				1.17	1.04	2100 (57.1%)	631 (17.1%)	368 (10.0%)	227 (6.2%)	354 (9.6%)	2731 (74.2%)	high

Accuracy Percentage (AP); error range within 1.5 mm was considered accurate.

The degree of accuracy was defined as very high (AP>90%), high (70%<AP<90%), medium (50%<AP<70%), low (50%<AP).

Table 2. Comparison of mean errors among the four time-points (T0, T1, T2 and T3) and between two time-points [(T0, T1) vs. (T2, T3)]

Compartment			Landmark	Initial stage (T0)		Pre-surgery stage (T1)		Post-surgery stage (T2)		Debonding stage (T3)		Comparison of mean errors
				Initial stage (T0)		Pre-surgery stage (T1)		Post-surgery stage (T2)		Debonding stage (T3)		among T0, T1, T2 and T3 stages ^a		(T0, T1) vs. (T2,T3) ^b
				Mean (mm)	SD (mm)	Mean (mm)	SD (mm)	Mean (mm)	SD (mm)	Mean (mm)	SD (mm)	P-value	Tukey HSD	P-value	Contrast matrix with the MANOVA
Skeletal landmark	Cranial Base		Nasion	0.58	0.42	0.59	0.48	0.55	0.39	0.64	0.60	0.698		0.852
			Sella	0.48	0.23	0.43	0.19	0.41	0.20	0.50	0.27	0.155		0.986
			Porion	1.04	0.55	1.07	0.76	1.17	0.85	1.01	0.57	0.493		0.566
			Orbitale	1.19	0.88	1.15	0.88	1.39	1.40	1.10	0.76	0.454		0.618
			Basion	1.41	1.32	1.59	1.60	1.95	1.94	1.61	1.52	0.148		0.092
	Maxilla	anterior	ANS	1.07	0.78	1.22	0.97	1.78	1.22	1.49	0.87	0.003 **	(T0, T1, T3) < (T3, T2)	0.003 **	(T0, T1) < (T2, T3)
		anterior	A point	1.27	0.89	1.28	0.78	1.50	1.07	1.59	1.16	0.151		0.040*	(T0, T1) < (T2, T3)
		posterior	PNS	1.16	0.79	1.14	0.87	1.29	1.09	1.17	0.82	0.823		0.587
	Mandible	anterior	B point	1.00	0.97	1.01	0.61	1.29	1.24	1.31	0.91	0.142		0.008**	(T0, T1) < (T2, T3)
		anterior	Pogonion	0.66	0.48	0.80	0.72	0.82	0.69	0.86	0.77	0.277		0.260
		bottom	Menton	0.83	0.52	0.70	0.39	0.74	0.38	0.82	0.45	0.298		0.786
		posterior	Articulare	0.76	0.39	0.75	0.42	0.73	0.40	0.84	0.58	0.540		0.616
Dental landmark	Maxillary dentition	anterior	Mx1C	0.48	0.37	0.49	0.55	0.41	0.26	0.38	0.18	0.355		0.096
		anterior	Mx1R	1.83	1.24	1.37	1.14	1.56	1.02	1.54	1.17	0.166		0.714
		posterior	Mx6D	1.66	1.18	1.63	1.31	1.20	0.80	1.23	0.88	0.032 *	(T2, T3, T1) < (T3, T1, T0)	0.008 **	(T2, T3) < (T0, T1)
		posterior	Mx6R	1.89	1.15	1.65	1.08	1.57	1.03	1.60	1.05	0.349		0.194
	Mandibular dentition	anterior	Md1C	0.48	0.45	0.62	1.10	0.47	0.39	0.37	0.31	0.215		0.096
		anterior	Md1R	1.77	1.22	1.52	1.00	1.37	0.96	1.63	0.95	0.303		0.400
		posterior	Md6D	2.15	1.79	1.71	0.91	1.51	1.14	1.33	0.75	0.008 **	(T3, T2, T1) < (T1, T0)	0.003 **	(T2, T3) < (T0, T1)
		posterior	Md6R	2.21	1.64	2.08	1.36	1.94	1.25	1.89	1.12	0.579		0.242
Total				1.20	1.10	1.14	1.02	1.18	1.09	1.15	0.95	0.376		0.895

^a Repeated measures analysis of variance (ANOVA) test was performed with post hoc test (Tukey HSD)

^b Repeated measures multivariate analysis of variance (MANOVA) test

T0, initial stage; T1, pre-surgery stage [presence of orthodontic brackets (OBs)]; T2, post-surgery stage [presence of OBs and surgical plates and screws (S-PS)], and T3, debonding stage [presence of S-PS and fixed retainers (FR)].

Table 3. Comparison of mean errors in each time-point (from T0 to T3) between the genioplasty and non-genioplasty groups

^a Repeated measures analysis of variance (ANOVA)

^bIndependent t-test was performed.

*, P<0.05

Table 4 Composition of the training, validation, and test dataset

Institution
Institution		Training set	Validation set	Test set	sum
A	Seoul National University Dental Hospital	1292	100	52	1,444
B	Kyung Hee University Dental Hospital	607	100	48	755
C	Kyungpook National University Dental Hospital	133	30	20	183
D	Asan Medical center	144	32	24	200
E	Ewha University Medical center	116	20	12	148
F	Wonkwang University Dental Hospital	95	26	8	129
G	Ajou University Dental Hospital	84	20	12	116
H	Korea University Anam Hospital	62	25	0	87
I	Chonnam National University Dental Hospital	48	16	8	72
J	Chosun University Dental Hospital	45	9	0	54
Total	lateral cephalograms	2626	378	184	3,188
Total	Class III patients	751		46	797

Table 5 The definition of cephalometric landmarks

Compartment			Landmarks	Description
Skeletal landmark	Cranial Base		Nasion (N)	The most anterior point on the frontonasal suture in the midsagittal plane
			Sella (S)	Center of the Sella Turcica
			Porion (Por)	The most superior point of the external auditory meatus
			Orbitale (Or)	The most inferior point of the orbital cavity contour
			Basion (Ba)	The most posterior and inferior point of the occipital bone
	Maxilla	anterior	ANS	The tip of anterior nasal spine
		anterior	A point	The deepest point between ANS and the upper incisal alveolus
		posterior	PNS	The most posterior point of the hard palate
	Mandible	anterior	B point	The deepest point between Pogonion and the lower incisal alveolus
		anterior	Pogonion (Pog)	The most anterior point on the symphysis
		posterior	Articulare (Ar)	Intersection between the inferior cranial base surface and the posterior surface of condyle
		bottom	Menton (Me)	The most inferior point on the symphysis
Dental landmark	Maxillary dentition	anterior	Mx1C	Crown tip of the maxillary central incisor
		anterior	Mx1R	Root apex of the maxillary central incisor
		posterior	Mx6D	Distal contact point of the maxillary first molar
		posterior	Mx6R	Distobuccal root apex of the maxillary first molar
	Mandibular dentition	anterior	Md1C	Crown tip of the mandibular central incisor
		anterior	Md1R	Root apex of the mandibular central incisor
		posterior	Md6D	Distal contact point of the mandibular first molar
		posterior	Md6R	Distal root apex of the mandibular first molar

No competing interests reported.

Accuracy of Artificial Intelligence-Assisted Landmark Identification in Serial Lateral Cephalograms of Class III Patients Who Underwent Two-Jaw Orthognathic Surgery

Status:

Version 1

Abstract

Figures

Introduction

Results

Discussion

Conclusions

Methods

Declarations

References

Tables

Additional Declarations

Status:

Version 1