This study acknowledges the inherent limitations and potential flaws of the application of AI in the field of medicine, demonstrating the sophisticated architecture and reliability of an IA model via external validation. Among numerous studies employing AI algorithms, only 6.0% of recently published research on the diagnostic analysis of medical images has reported external validation performance33. The IA model, developed in this study using a two-stage binary algorithm combined with transfer learning with fine-tuning, embeds parameters learned from a high-quality labelled dataset of a large population across numerous epochs. The model simplifies the role of the clinician to merely activating it with the push of a button at the point of DICOM sorting from patient CT scans, which subsequently provides diagnostic probabilities for the input images.
However, we remain cautious about overestimating the capabilities of the IA model, which are specifically modified for feature extraction of the appendix within the VOI. The specialisation may limit its generalisability and reproducibility for patients with abdominal pain in ERs; however, it is expected to offer considerable benefits to clinicians and patients in medically underserved regions without radiologists available to read CT images43.
In the field of medical image analysis, TL facilitates knowledge transfer at the parametric level and is extensively used to optimise ML models. It enhances the learning time and performance by leveraging datasets from the source domain, addressing the lack of data annotations in the target domain and the unavailability of large labelled datasets29. However, we acknowledge that clinicians, who integrate patient history, physical examination, review of systems, and laboratory results to formulate a diagnosis before interpreting CT images, do not strictly adhere to the two-stage binary decision-making process inherent in TL. Nonetheless, the rationale for not employing multiclass algorithms involves emulating human cognitive processes as closely as possible. However, it does not imply that clinicians cannot diagnose complicated appendicitis without prior identification of normal or simple cases. Instead, TL facilitates a more efficient learning process for interpreting complicated appendicitis cases by adapting pre-trained tasks, as opposed to starting the ML process from scratch30.
In this study, the VOI provided a feature map that enabled spatial perception of areas beyond the appendix, particularly the extra-appendiceal region. Utilising handcrafted annotations focused solely on the appendix and mesoappendix can result in information loss concerning extra-appendiceal features, such as extraluminal gas, fluid collection, appendicolith, bowel content due to perforation, and adjacent structural changes caused by appendiceal inflammation. Notably, less than 3% of the true class cases were located outside the VOI, underscoring the challenge of expanding the ROI, which may detrimentally affect the performance of the model. The range of the VOI, established to cover all types of appendicitis locations, including pelvic types, was optimised by adjusting the upper and lower margins via EDA37,38.
However, fatal errors are inevitable when analysing data with potential causes outside of the VOI—a situation applicable to data from patients excluded in real-world clinical settings. Despite being rare, oncologic risks are concerning within the VOI field. Masquerading appendicitis harbouring hidden malignancies, particularly those complicated by perforation, presents a diagnostic challenge and can worsen patient outcomes44. An accurate preoperative diagnosis of complicated appendicitis with potential tumour risk is difficult45.
In this study, the most common cause of false positive errors in both the development and external datasets was ascending colon diverticulitis, followed by terminal ileitis, regardless of the 3D-CNN model employed. Similar to the issues faced by clinicians, the IA model did not effectively address the challenge of interpreting CT images of patients with appendicitis that mimics ascending colon diverticulitis or terminal ileitis46,47. An analysis of Grad-CAM images from false class cases revealed that inflammation might alter image texture features, such as edgeness per unit area and local neighbourhood intensity, involving the appendix and surrounding extra-appendiceal structures, such as fat, fluid, or bowel within the VOI48. However, the model failed to distinguish discrepancies between salience and activation maps, lacking top–down evidence for the textures of false error images49. The technical key to reducing false errors lies in the incorporation of additional data, balanced fitting, and optimising the loss function30.
This study acknowledges several limitations, including potential iatrogenic perforation due to surgical manipulation and variable responses to antibiotics or delayed surgery between the times of radiologic and pathologic labelling. Despite these limitations, we posit that our robust dataset, which includes data from normal individuals, patients with non-specific conditions, and patients with a broad spectrum of diseases, may yield better results than those of previous studies3,22 that lacked control data. However, issues such as the automatic uploading of sorted DICOM series from the PACS platform, large amount of excluded data, limitation of DL to the confined VOI rather than the entire abdominal section, and hidden risk of appendiceal tumours within the VOI suggest that practical application in real-world settings may still be distant. To address the limitations of the model, we are currently in the process of conducting a prospective randomised study comparing the diagnosis by non-radiologists and the IA model [ClinicalTrials.gov ID: NCT06175169] to ensure that the model fulfils ethical requirements and is safe.
In conclusion, this study demonstrated that the proposed DL model can identify non-appendicitis, acute appendicitis, and complicated appendicitis from CT scan images of critical patients who presented right-sided or lower abdominal pain in the ER, representing a diagnostic approach for clinicians. The IA model, owing to its generality and reproducibility within the VOI, may alleviate the burden of CT interpretation for physicians, particularly in medically vulnerable systems or regions that lack readily available radiologists or specialist-based interpretations.