Gliomas are the most common types of glial-based primary tumors in adults, which are specified with several malignancy grades, namely grade I (pilocytic astrocytoma, gangliocytoma, and ganglioglioma), grade II (astrocytoma, oligodendroglioma, and oligoastrocytoma), grade III (anaplastic astrocytoma, anaplastic oligodendroglioma, and anaplastic ependymoma), and grade IV (glioblastoma) according to World Health Organization (WHO) (Ahmed, Oborski, Hwang, Lieberman, & Mountz, 2014; Ranjbarzadeh et al., 2021; Tiwari, Srivastava, & Pant, 2020).
Grades I and II of gliomas usually are known as benign or low-grade brain tumors; however, grades III and IV are more aggressive and specified as malignant or high-grade tumors (Ghaffari, Sowmya, & Oliver, 2020). Despite the recent advances in therapeutic and diagnostic methods for malignant gliomas, the median survival is less than five years for patients with anaplastic glioma (grade III) and 15 months for patients with glioblastoma (GBM) (Ahmed et al., 2014). Gliomas represent about 80% of all malignant brain and central nervous system (CNS) tumors. More than 50% of these gliomas are diagnosed as GBM (Barnholtz-Sloan, Ostrom, & Cote, 2018). In the United States, the annual incidence of all brain tumors, gliomas, and GBMs per 100,000 population between 2010 to 2014 were 22.64, 5.74, and 3.20, respectively (Barnholtz-Sloan et al., 2018).
The neuroimaging data provide precious information related to the shape, size, location, and metabolism of brain tumors for clinicians, which can be used to assess the status of the tumor before and after therapeutic intervention (Magadza & Viriri, 2021). Magnetic resonance imaging (MRI) sequences such as T1-weighted (T1w), T2-weighted (T2w), T1-contrasted (T1c), and fluid attenuation inversion recovery (FLAIR) provide substantial contrast for various brain tissues, which can be used to discriminate the different parts of tumor and normal tissues (Saha & Panda, 2018; Wadhwa, Bhardwaj, & Singh Verma, 2019).
Brain tumor segmentation for locating and assessing the tumor region and tumor classification for recognizing its grade are essential steps for choosing the proper treatment plan and effectiveness of treatment. As a result, physicians usually perform these procedures manually before starting the treatment plan; in the meantime, manual segmentation and classification of tumors are laborious and time-consuming tasks and different specialists may have varying diagnoses, which may reduce treatment effectiveness. (Ranjbarzadeh et al., 2021).
Considering the reasons mentioned above, automatic brain tumor segmentation could prepare valuable morphological information for clinicians about different tumor parts, including core, enhanced, and whole tumor regions, leading to timely and proper diagnosis and treatment of brain tumors (Ranjbarzadeh et al., 2021; Saman & Jamjala Narayanan, 2019). In recent decades, machine learning (ML) advances have led to increasing interest in automatic medical image analysis. However, traditional ML approaches mainly require prior knowledge and a manual feature extraction process, which can be time-consuming. As a new subcategory of ML, deep learning recently showed major benefits in overcoming the limitations of traditional ML approaches (Fathi, Ahmadi, & Dehnad, 2022). More complex and high-level features can be extracted automatically and then given to a deep learning-based classification or segmentation algorithm, which means that the feature extraction and classification/segmentation steps are merged in deep learning (Fathi et al., 2022; Havaei et al., 2017). Among various deep learning approaches, convolutional neural network (CNN) architectures have shown superior results, especially in detecting and analyzing neurological diseases (Fathi et al., 2022; Valliani, Ranti, & Oermann, 2019).
In this study, a CNN-based architecture called U-Net has first applied to the BraTS 2018 dataset for automatic brain tumor segmentation to determine the different parts of the tumor, namely core, enhanced, and whole tumor regions. Then, another CNN architecture called VGG16 was trained to classify the images into low and high grades (LGG/HGG) tumors. The transfer learning parameter initialization method was used to decrease the learning time and enhance the performance of the model. Evaluating the applicability of the model was done by gathering and using a local dataset to assess the trained models. During the final step, the volume of the tumor regions before and after radiotherapy on cases in the local dataset was measured and statistically compared to assess the efficiency of the treatment.
To the authors' best knowledge, the assessment of treatment responses pre and post-radiotherapy in glioma patients has been mentioned in none of the reviewed studies. Hence, due to the importance of assessing treatment response in these patients, we evaluated that in glioma patients who underwent 3D conformal radiotherapy about 3 to 8 months after. Hence main contributions of this study are described as follows:
- A multimodal approach including four primary MRI sequences, namely T1w, T2w, T1c, and FLAIR, was utilized to enhance the accuracy of segmentation and classification procedures.
- The segmentation process was done on both LGG and HGG cases.
- The final segmentation and classification models were applied to a local dataset to assess their practical applicability.
- The proposed models have been conducted on the local dataset before and after radiotherapy intervention aiming to measure the effectiveness of treatment statistically.
1.1 Related works
In this section, we glance at similar studies and briefly review their automatic brain segmentation method. Brain tumor segmentation (BraTS) challenges, which have been held annually since 2012, have inspired the application of ML approaches in this field. The most popular methods used in the early years of the BraTS challenge were based on traditional ML approaches such as Random Forest (RF), logistic regression, Markov Random Field (MRF), and Conditional Random Field (CRF) (Ghaffari et al., 2020). Despite the unsatisfying performance of the traditional methods, they opened new ways for automatic brain tumor segmentation. Later advances in the computational power of computers and the ML approaches made deep learning-based methods more popular in this field. Concerning this, convolutional neural networks (CNN) were used in brain tumor segmentation for the first time in 2014.
Urban et al. in 2014 used a simple 3D-CNN architecture with three convolutional layers as a voxel-based classification method to classify edema, non-enhancing tumor, enhancing tumor, necrosis, air, and normal tissue in the multimodal images and obtained fair results for whole, core and enhancing tumor segmentation (Urban, Bendszus, Hamprecht, & Kleesiek, 2014). In 2015, Havaei et al. proposed a 2D-CNN architecture called InputCascadeCNN model for brain tumor segmentation. They achieved Dice scores of 0.88, 0.79, and 0.73 for the whole tumor, tumor core, and enhancing tumor, respectively (Havaei, Dutil, Pal, Larochelle, & Jodoin, 2015). In 2016, a new CNN-based architecture called DeepMedic was introduced by Kamnitsas et al., which comprised 11 3D convolution layers with residual connections to obtain a more efficient model. They reported Dice scores of 0.89, 0.76, and 0.72 for whole, core, and enhancing tumor regions, respectively (Kamnitsas et al., 2016).
In recent years, the number of studies using CNN-based models, especially U-Net architecture, has increased dramatically. Chen et al. applied a novel separable 3D U-Net architecture on BraTS2018 and reached the Dice scores of 0.69, 0.84, and 0.78 for enhancing tumor, whole tumor, and tumor core, respectively (Chen, Liu, Peng, Sun, & Qiao, 2019). In another submitted study to BraTS2018, Feng et al. (Fang & He, 2018) used three 3D U-Nets with different hyperparameters and combined them via simple averaging of the probability of all classes obtained by each model. They reported Dice scores of 0.9, 0.83, and 0.78 for the whole tumor, tumor core, and enhancing tumor, respectively (Fang & He, 2018). Similar to the previous study, Caver et al. also used three different U-Nets to segment brain tumors, with the difference that each model has been employed for segmenting one region of interest. The Dice scores for the whole tumor, tumor core, and enhancing tumor were 0.87, 0.76, and 0.72, respectively (Caver, Chang, Zong, Dai, & Wen, 2018).
Kermi et al. proposed a 2D U-Net architecture to segment the whole and other intra-tumor regions. In order to address the class imbalance issue, they have used novel loss functions called Weighted Cross Entropy (WCE) and Generalized Dice Loss (GDL). They achieved the Dice scores of 0.86, 0.80, and 0.76 on validation data for the whole tumor, tumor core, and enhancing tumor, respectively (Kermi, Mahmoudi, & Khadir, 2018). In Naser et al.'s study, the closest one to our study, a multi-task deep learning-based method has been proposed to segment and classify grades II and III of gliomas. The segmentation and classification tasks were carried out by U-Net and VGG16 architectures, respectively. The reported Dice score and tumor detection accuracy were 0.84 and 0.92, respectively (Naser & Deen, 2020).