Data
We queried the radiation oncology patient database of the Dana-Farber Cancer Institute and Brigham Women’s Hospital for any radiation planning structure sets with an esophagus structure segmented as part of routine clinical care. 574 patients with predominantly lung cancer were selected, each of which had a 3D CT image and an expert-approved esophagus segmentation saved in the respective DICOM RTSTRUCT file. We grouped images of patients based on their corresponding attending physician and selected the top 8 physicians with the most CT scans, resulting in a final study cohort of 394 patients. The study was conducted under a protocol approved by the Dana-Farber/Harvard Cancer Center institutional review board. All methods were performed in accordance with the relevant guidelines and regulations. The study was conducted under a waiver of informed consent approved by institutional review board (protocol Dana-Farber/Harvard Cancer Center 11–286).
Training and Test Sets
The lengths of segmented esophagus varied significantly in the real-world data generated by clinicians due to variation in clinical practice depending on radiation therapy technique with full length of the esophagus segmented for conventionally fractionated RT to calculate volumetric exposure (e.g. volume of esophagus receiving 60 Gy or more) versus partial esophageal segmentation at level of the tumor for stereotactic body radiation therapy cases in which point maximum doses to the esophagus are evaluated clinically. Thus, we split the data into full and partial esophagus in terms of length by setting a threshold (L = 18 centimeters). We picked this length because two physicians (Physician 1&6, MD1&6 Model) who tend to generate more complete segmentations of esophagus per national guidelines (RTOG) [18]. As a result, we ensured enough full-length esophagus cases to train multi-MD models. To have a benchmark model performance, we trained a model with multi-physician data (MD1-7 Model). We also trained a model by excluding Physician 7 as we observed more edge roughness in these cases in our primary quality check (MD1-6 Model). Moreover, we also needed to compare model performances among individual-physician models (MD7 Model, MD6 Model, and MD1 Model).
In the end, we constructed six separate models, each of which had the corresponding training data, and the remaining data were used for validation (Fig. 7). Three models were trained and tuned using various combinations of segmentations from multiple physicians (MD1-7 Model, MD1&6 Model, and MD1-6 Model), and the other three of which were trained using images and segmentations from individual physicians (MD7 Model, MD6 Model, and MD1 Model). MD1&6 Model was trained using the combination of full-length segmentations from Physician 1 and Physician 6 (n = 81). MD1-7 Model is composed of training data used in MD1&6 Model as well as randomly sampled segmentations from other five physicians (n = 142). MD1-6 Model was built using data in MD1-7 Model except for the exclusion of Physician 7’s (n = 122) cases. MD7 Model, MD6 Model, and MD1 Model were physician-style-adapted models trained using full-length cases segmented by Physician 7 (n = 36), 6 (n = 35), and 1 (n = 46), respectively. The test set included the segmentations of physician 8 who was intentionally excluded from the training data to have an independent validation dataset, and any data not used in the training set for a given model.
Deep-learning Models for Automatic Segmentation
We developed a deep-learning semantic segmentation system to automatically localize and segment the esophagus from a CT scan and using expert segmentations generated during clinical care as ground truth training data. The proposed system consists of two steps (i.e., localization and segmentation), each of which used a 3-dimensional U-Net architecture with residual blocks as backbone (Fig. 1). The encoder-decoder structure follows the original implementation [19]. The residual block consists of two residual layers that involve batch normalization, 3D convolution, and ReLU activation (Fig. 1). In the step of localization, we reduced the size of the original 3D CT volume and segmentation and used them to train a localizer to identify the rough location of the esophagus [20]. We then mapped the compressed volume back to its original size and acquired the predicted coordinates of the mapped esophagus.
After resampling the original volume and segmentation with a voxel spacing of 1 mm, we applied the predicted coordinates onto the resampled volume to localize the esophagus and expanded the bounding box to the size of 128 x 128 x 128 pixels. To increase the contrast between the target structure and neighbors and speed up training, we applied a CT window (-340 < HU < 460), followed by intensity normalization, to the selected sub-volume that contains the esophagus and the surrounding tissue.
Data augmentation methods were adopted to enrich the complexity of training data, including random flipping, rotation, translation, and scaling. Separate models with different combinations of images and segmentations were trained and tuned using the residual U-Net. Predicted segmentations were generated and processed to remove small fragments. Final predictions were evaluated by volumetric Dice Coefficient. All models were trained on a NVDIA GPU with 32 GB of memory, using TensorFlow 2.2. The batch size was set to 4, the learning rate was 0.01 and the validation split was 0.2.
Accuracy Assessment
The accuracy of AI segmentation of the esophagus was evaluated by using the volumetric Dice coefficient with physician’s segmentations as ground truth. The volumetric Dice coefficient [21] is defined as:
Dice Coefficient\(=\frac{2\left({V}_{p}\cap {V}_{GT}\right)}{{V}_{p}+{V}_{GT}}\)
where \({V}_{p}\) represents the predicted volume by AI models and \({V}_{GT}\) represents the ground truth segmentation volume created by physicians. Since we trained models using full-length segmentations, for partial-length segmentations, we only evaluated the volumetric Dice only within regions where ground truth labels existed.
Statistical Analysis
P-values for the comparison of edge roughness, length of segmentations, and Dice coefficients were generated from the Dunn’s test, a non-parametric pairwise multiple comparison test following the rejection of an ANOVA null hypothesis [22, 23]. Proportions in Dice outliers were compared using Fisher exact test [24]. P-values were two-sided and considered significant below 0.01.