A deep learning-based approach for Multiple Sclerosis Lesion Segmentation

doi:10.21203/rs.3.rs-5303677/v1

Purpose: Multiple Sclerosis (MS) is a chronic disease of the Central Nervous System (CNS), characterized by the presence of disseminated lesions in the brain and Spinal Cord (SC). Magnetic Resonance Imaging (MRI) has become an essential tool for studying the anatomy and functions of the CNS in vivo, enabling not only the identification of brain structures but also the detection of damaged tissue in various neurodegenerative diseases, including MS. The segmentation of lesions on MR images is a crucial step in the diagnosis and monitoring of the disease. However, manual segmentation of MS lesions is a complex and time-consuming task requiring considerable expertise.

Methods: This paper proposes a fully automated method for MS lesion segmentation based on a Convolutional Neural Network (CNN) architecture. The model was trained on datasets from the MICCAI 2016 and ISBI 2015 international challenges. FLAIR images from these databases were used as input to the CNN.

Results: The results show a significant improvement in the accuracy and robustness of the model, resulting in high-quality segmentation of MS lesions. The model achieved remarkable performance, with a Dice Similarity Coefficient (DSC) of over 89%, outperforming recent methods.

Conclusion: These promising results underline the considerable potential for future advances in the automated segmentation of MS lesions.

Multiple Sclerosis

MRI

ISBI 2015

MICCAI 2016

lesion segmentation

CNN

Multiple Sclerosis (MS) is a chronic inflammatory neurological disease of the Central Nervous System (CNS) that affects around 2.8 million people worldwide [1, 2]. Generally, MS affects people between the ages of 20 and 50 [3]. The disease causes inflammatory and demyelinating lesions in the brain and Spinal Cord, resulting in a range of neurological symptoms. These symptoms include visual disturbances, motor coordination problems, genital sphincter dysfunction, cognitive difficulties such as concentration and memory problems, as well as persistent fatigue [4, 5]. The diagnosis of MS represents a major clinical challenge due to its complexity and clinical variability. However, thanks to technological advances, in particular Magnetic Resonance Imaging (MRI), this diagnosis has improved considerably over the years. MRI plays a crucial role in this process, allowing detailed visualization of the brain and SC lesions, characteristic of the disease [6–8]. MRI modalities, such as T1-weighted (T1-w), T2-weighted (T2-w), Proton Density (PD), Fluid-Attenuated Inversion Recovery (FLAIR), and Gadolinium- enhanced T1 (T1-Gd), provide valuable information on the location, size and progression of lesions [9–12], making it easier to confirm the diagnosis and assess disease progression in patients. However, despite these advances, diagnosing MS remains a complex process, requiring in-depth clinical expertise and meticulous interpretation of images by radiologists. In this context, the integration of Artificial Intelligence (AI), in particular Machine Learning (ML) and Deep Learning (DL), offers new prospects for improving the accuracy and diagnosis of MS [13, 14–16, 18, 19]. AI enables automated and accurate analysis of MRI images, facilitating early diagnosis and monitoring of MS. This innovative approach opens the way to more personalized treatment strategies and better management of patients suffering from this complex neurological disease [17, 20]. In this paper, we present a DL-based approach for the automatic segmentation of MS lesions from MR images. By exploiting the advantages of AI, our approach aims to increase diagnostic accuracy, reduce evaluation times and facilitate earlier for a personalized management of MS patients. The remainder of the article is organized as follows: the second section presents a review of the literature on research aimed at detecting MS lesions based on DL methods. The third section presents the method adopted to achieve the objective, describing the proposed method and the dataset used. The fourth section presents the results obtained. Finally, a discussion and a general conclusion summarizes our proposed method and our future prospects.

AI has been driving a major revolution in the medical field for a number of years now, fundamentally transforming approaches to the diagnosis and management of various diseases [21–25]. This revolution has a significant impact in the field of neurological diseases, particularly in the diagnosis of MS [14]. Using MRI images and DL, it is now possible to precisely segment MS lesions, providing results of crucial importance for the diagnosis and management of this disease. These advances enable healthcare professionals to obtain an in-depth understanding of the extent of damage and to plan appropriate interventions. In this context, a variety of studies and research have explored different approaches, highlighting the constant evolution of methodologies to improve the detection and understanding of this complex disease [15, 26, 27]. Some of these initiatives have opted for ML methods for their diagnostics [14, 16]. At the same time, other research has turned to DL techniques to meet the challenges of segmenting and diagnosing MS lesions. In particular, CNNs have demonstrated exceptional results in the field of brain imaging, notably in the segmentation of MS lesions [17, 18]. The work of [18, 17, 28] explored the use of CNNs for precise segmentation of lesions on MRI images. These DL-based approaches have demonstrated their ability to process complex data, opening up new prospects for more accurate diagnoses and more targeted interventions. Furthermore, [26, 27, 29, 30] have proposed approaches based on the U-net architecture for the segmentation of MS lesions from MRI images. For example, in [26], the ‘Minimally Parameterized U-net (MPU-net)’ model based on the U-Net architecture was used to segment MS lesions from FLAIR modality. The process adopted consists of three parts: an initial pre-processing stage followed by data augmentation, and a segmentation phase based on the proposed model. On the other hand, Afza et al. [18] proposed a DL method based on CNNs, where 2 CNNs per patch are exploited for more precise lesion segmentation. The authors of this research aim to train their models on T1-w, T2-w and FLAIR multimodal MRI images for better assessment. In the majority of studies, Adam optimizer is widely used for MS lesion segmentation [18, 28, 31–36], due to its ability to dynamically adapt the Learning Rate (L_r) based on gradients and optimize model convergence speed. Typically, it is applied with a L_r varying between 10^− 4 and e^-4 [27, 28, 35], allowing precise adjustment of the parameters over the iterations. The most commonly used batch sizes are 32 and 128 [18, 28, 32, 37], which ensures a compromise between training stability and computational efficiency. Generally, training spread over 200 epochs [27, 35–37] to guarantee stable and robust convergence. In addition, Dice loss and Cross Entropy (CE) are the most frequently used functions in MS lesion segmentation [27, 28, 32, 37], each having specific advantages for class imbalance management and segmentation accuracy improvement. In other research [33], these two functions have been combined to create a hybrid function called ‘Combo Loss’ (Loss_comb), which has shown remarkably promising results in improving both model accuracy and generalization in the context of MS lesion segmentation. This type of approach adapt model to the complex variations in brain structures and to different data sets. Today, the availability of data sets presents a challenge in the diagnosis of MS disease. Some researchers have used the public “International Symposium on Biomedical Imaging 2015” (ISBI 2015) dataset [26–28, 31]. Others have based their research on the “Medical Image Computing and Computer Assisted Intervention” (MICCAI 2016) dataset [29, 32–37]. These datasets offer free accessibility with a various MRI modality such as: T1-w, T2-w, FLAIR, PD and T1-Gd. To evaluate the performance of the proposed method, Dice similarity coefficient, Precision named as Positive Predictive Accuracy (PPV), Sensitivity referred as True Positive Rate (TPR), Volume Difference (VD), and the Intersection Over Union (IOU) are frequently used in the literature for the evaluation of MS lesion segmentation results [17, 18, 26, 28, 32, 36, 37].

The proposed approach is based on three fundamental pillars. First, a data preparation step is performed consisting of a pre-processing stage followed by a data augmentation phase to enrich the database, followed by a MS lesion segmentation step using a DL model. Figure 1 illustrates the general workflow of the proposed segmentation method.

A. Dataset

In this work, two public datasets ISBI 2015 and MICCAI 2016 were used.

ISBI 2015 dataset

The ISBI 2015 dataset [38] consists of 19 subjects divided into 5 training sets and 14 test cases. All data were segmented by two experienced evaluators with over 2 years of experience. Only the masks from the training set are publicly accessible, while those from the test set are masked for the challenge. Each subject in the training set has an average of 4.4 time points, and the test set also has an average of 4.4 time points per subject. The clinical characteristics of the subjects, as well as the imaging parameters for each dataset, are detailed in Table 1.

Table 1

Summary of clinical and imaging characteristics of the ISBI 2015 dataset.
Dataset	Number of subjects	Clinical characteristics
Dataset	Number of subjects	Genre	Age (Mean ± Std)	Time-points (Mean ± Std)	Modality	Voxel size (mm³)
Training set	5	1 Male, 4 Females	43.5 ($\:\pm\:$10.3)	4.4 ($\:\pm\:$0.55)	MPRAGE PD-w T2-w FLAIR	0.82 x 0.82 x 1.17 0.82 x 0.82 x 2.2
Testing set	14	3 Males, 11 Females	39.3 ($\:\pm\:$8.9)	4.4 ($\:\pm\:$0.63)	MPRAGE PD-w T2-w FLAIR	0.82 x 0.82 x 1.17 0.82 x 0.82 x 2.2

The longitudinal datasets include MRI scans with T1, T2, PD, and FLAIR modalities, captured at 3 to 5 time points using a 3T MRI scanner. Each volume contains 18 2 slices, some of which are entirely black. The dimensions of each slice are 181 x 217 pixels, with a spatial resolution of 0.82 x 0.82 mm and slice thickness ranging from 1.17 mm³ to 2.2 mm³. Figure 2 illustrates an example of an MRI slice from T1-w, T2-w, PD, and FLAIR modalities, along with Ground Truth (GT) given by expert 1 for patient 1 from the ISBI 2015 training set.

MICCAI 2016 dataset

The MICCAI 2016 dataset [39, 40] was generated by participating neurologists as part of the “Observatoire Français de la Sclérose En Plaques” (OFSEP) and comes from four different MRI scanners from various manufacturers (Siemens, Philips, and GE). The MRI scanners included three Tesla magnets and 1.5 Tesla magnets.

Table 2

Summary of clinical and imaging characteristics of the MICCAI 2016 dataset.
	Patient age (Mean ± Std)	Patient genre	Patient number of lesions (Mean ± Std)	Patient lesions load (Mean ± Std)
Training dataset	46.92 (± 10.16)	8 Males, 30 Females	40.71 (± 39.48)	12350.68 (± 15028.476)
Testing dataset	41.6 (± 9.84)	7 Males, 8 Females	37.13 (± 27.06)	20756.42
All dataset	45.41 (± 10.26)	15 Males, 38 Females	39.69 (± 36.18)	14729.66 (± 17006.0636)

A total of 53 patients were recorded in this dataset, divided into 15 training cases and 38 test cases. Each patient received T1-w, T2-w, FLAIR, PD, and T1-Gd images, with dimensions ranging from 240 x 320 to 512 x 512 pixels. The spatial resolution of these images ranges from 0.43 x 0.43 mm to 0.70 x 0.74 mm, with slice thicknesses ranging from 0.7 to 3 mm³. Table 2 provides more details about the clinical characteristics of both training and testing datasets. For the training set, the slices were manually segmented by seven qualified junior radiologists. Figure 3 presents an example of slices from the MICCAI 2016 dataset.

B. Preprocessing

In this work, we utilized the preprocessed images of the ISBI 2015 and MICCAI 2016 datasets. To enhance the quality and consistency of the data, we conducted additional preprocessing steps. This included resampling the images to a standardized resolution of 192 x 192 x 192 voxels to ensure uniformity across the dataset. We have also normalized the intensities of the images to resolve problems of variability that could complicate the training of the DL model. Furthermore, we meticulously removed any undesirable black images which could potentially affect the accuracy of our analysis.

C. Data Augmentation

To optimize the performance and robustness of the NN model and to increase both the amount and diversity of available data, we applied various data augmentation techniques. Using the image data augmenter generator, we applied a set of geometric transformations to each training image, including a random rotation of 90°, a vertical flip, an offset, a scaling, and a grid distortion. These manipulations are intended to enrich the training dataset and to adapt the model to a wider variety of situations. Figure 4 shows an example of slices of the ISBI 2015 dataset after the application of data augmentation techniques.

D. Proposed CNN Architecture

The proposed CNN architecture for MS lesion segmentation is inspired by the U-Net architecture introduced by Ronneberger et al. for medical image segmentation [41]. It incorporates a ResNet-34-based encoder coupled with a U-Net decoder. This approach leverages the robustness of ResNet's residual blocks to extract deep features while ensuring the preservation of memory flow throughout the network [42]. The considered model consists of two main paths: a contraction path or the Encoder (E) with residual blocks and an expansion path or named Decoder (D) as presented in Fig. 5.

Table 3 provides a comprehensive overview of the CNN architecture used in this study, detailing the specific configurations at each stage of the model and highlighting the hyperparameters chosen to optimize network performance for accurate lesion segmentation. For the encoder, intermediate and decoder blocks, the number of residual sub-blocks, the types of layers used, the number and the size of filters at each level are detailed.

Table 3

Proposed CNN details
	Block	Number of residual subs-block	Type	Filter Size/Number		Pad size	Stride
Encoder	Input layer	-	1 Conv2D	7 x 7 x 64		1	2
	Input layer	-	1 Max Pooling	2 x 2		-	2
	Encoder Block 1	3	[2 Conv2D] x 3	3 x 3 x 64		1	1
	Encoder Block 2	4	[2 Conv2D] x 4	3 x 3 x 128
	Encoder Block 3	6	[2 Conv2D] x 6	3 x 3 x 256
	Encoder Block 4	3	[2 Conv2D] x 3	3 x 3 x 512
Intermediate Block	-	-	2 Conv2D	3 x 3 x 1024
Decoder	Decoder Block 1	-	Conv2D Transpose 2 Conv2D	3 x 3 2 x 2	x 512	1	1
	Decoder Block 2	-			x 256
	Decoder Block 3	-			x 128
	Decoder Block 4	-			x 64
	Final layer	-	1 Conv2D	1 x 1	x 2

a. Encoder: ResNet-34 backbone: In our model, the encoder is a modified version of ResNet-34, specifically adapted to extract relevant features from brain MRI images of MS patients. It consists of four residual blocks. In the first stage, a 7x7 convolution with a stride of 2 and padding of 1 is firstly applied followed by ReLU activation, Batch Normalization (BN), and 2x2 MaxPooling. In Fig. 5, $\:i\:$represents the index of encoder and decoder blocks, while $\:i=\{\text{1,2},\text{3,4}\}$. Each Encoder block (E_i) is composed of $\:j\:$Residual sub-blocks (R_j), where $\:j$ takes the values {3,4,6,3} respectively for considered encoder block i; $\:i=\left\{\text{1,2},\text{3,4}\right\}$. Each R_j includes two successive 3x3 convolutional layers with k filters, $\:k=\left\{\text{64,128,256,512}\right\}$ fixed according to the decomposition level $\:i=\left\{\text{1,2},\text{3,4}\right\}$ respectively. Each layer is followed by a ReLU activation function and a BN layer. An identity mapping is finally added to enhance the learning of features at different layers.

b. Intermediate Block: The intermediate block bridges the encoder and decoder blocks. It consists of two successive 3x3 convolutional layers, each followed by ReLU activation function and BN. This block prepares the features for reconstruction in the decoder.

c. U-Net Decoder: The U-Net decoder recovers spatial details lost during the encoding phase and reconstructs a segmentation map with the same resolution as the input image. The decoding process consists of four blocks. Each Decoder block (D_i) begins with an Up-sampling operation, typically performed by a 3x3 Conv2DTranspose layer, which doubles the spatial dimensions of the reduced feature map. At each decoding level, skip connections are employed to concatenate the corresponding encoder features (E output), merging global contextual information with fine details. This is followed by two successive 3x3 convolutional layers, each one is paired with ReLU activation and BN as illustrated by the first Decoder block in Fig. 5. This process is repeated across all four levels. Finally, a 1x1 convolution adjusts the depth of the feature map to match the number of classes to be segmented, producing the final segmentation map.

E. Loss Function

In this study. we adopted Loss_comb as the loss function, an innovative approach resulting from the judicious combination of Dice loss with modified CE [43]. The choice of this combination function stems from its clear advantages in solving the problem inherent in the imbalance between input classes, particularly when segmenting small lesions against a large context or background. This approach offers the ability to simultaneously control the trade-off between FP and FN, while facilitating smooth training through the use of CE. The mathematical formulation of Loss_comb is defined by the following Equation:

Loss _comb = 𝛼. 𝐷𝑖𝑐𝑒 𝐿𝑜𝑠𝑠 + 𝛽. 𝐶𝐸 (1)

Where α and β are respectively the weights assigned to Dice loss and CE. In Eq. 1, α controls the amount of Dice term contribution in the loss function. β ∈ [0. 1] controls the level of model penalization for FP/FN. In this work, α and β were fixed, empirically, equal to 0.5 to keep the model parameters out of bad local minima via the global spatial information provided by Dice [43].

A. Evaluation metrics

The evaluation of the model is carried out using commonly employed metrics defined as follows:

Dice Similarity Coefficient: is defined by the duplication of the area of overlap between the predicted and actual values, divided by the total number of pixels in the two images. Mathematically, it is expressed as follows:

$$\:DSC=\:\frac{2TP}{TP+FP+FN}$$

2

Sensitivity: represents the Rate of TP (TPR) in relation to the total number of positive annotations. It assesses the model's ability to correctly detect positive samples. Eq. 3 defines the TPR as follows:

$$\:Sensitivity=\:\frac{FP}{FP+TP}$$

3

Precision: is the TPR among the instances predicted to be positive. It measures the accuracy of the model's positive predictions, taking into account both TP and FP. It is calculated by Eq. 4:

$$\:Precision=\:\frac{TP}{FP+TP}$$

4

False Positive Rate (FPR): is the rate of FP in relation to the total number of negative annotations. It assesses the model's ability to avoid false discoveries in terms of false lesion detections. The FPR is calculated using the Eq. 5:

$$\:FPR=\:\frac{FP}{FP+TP}$$

5

Intersection Over Union (IoU): indicates the overlap between the predicted boundary box and the GT boundary box. It is calculated using Eq. 6:

$$\:IoU=\frac{TP}{TP+FN+FP}$$

6

Where, TP, FP and FN refer respectively to the number of pixels corresponding to lesions and correctly classified as lesions, the number of pixels not corresponding to lesions but incorrectly classified as lesions, and the number of pixels corresponding to lesions but incorrectly classified as non-lesions.

B. Experimental Results

In this work, we exploited FLAIR images from the ISBI 2015 and MICCAI 2016 training datasets, both accompanied by GT annotations essential for evaluating segmentation performance. The training process was carried out on a machine equipped with 128 GB of memory, running on an Intel(R) Xeon(R) processor [email protected] GHz (2 processors). The Python language was used because of its predominance in the field of DL. The dataset was divided into two subsets: 80% for training and 20% for validation. A series of experiments were carried out with various configurations of optimizers, L_r and loss functions, in order to optimize the model's performance. We obtained different values for the Dice, a key indicator for assessing the accuracy of the segmentations. Better performances, for both ISBI 2015 and MICCAI 2016 datasets, were obtained when the Adam optimizer, configured with a L_r of 4e^-5, an initial batch size of 32, and a fixed number of epochs of 200, were used to train the model.

Figures 7 and 8 show visual segmentation examples for three FLAIR MRI slices of MS patients, from the ISBI 2015 and MICCAI 2016 datasets. These results show accurate segmentation of lesions, particularly in complex and high-contrast areas. Furthermore, the variation curves of the Dice and the loss function, as a function of the number of epochs, presented in Fig. 9, highlight the stability and convergence of our model, even before 200 epochs. This stability is crucial to ensure that the model does not overlearn and remains generalizable to other data sets.

In addition, we evaluated the impact of data augmentation, a technique often used to improve the robustness of DL models. By comparing metrics with and without this technique, we found significant improvements in segmentation performance. Metrics value for both datasets are detailed in Table 4. The increase in data was particularly beneficial for the segmentation of small lesions, which are often difficult to detect due to their low contrast.

Table 4

Metrics values for both datasets. Values in bold indicates the obtained best value.
Database		Metrics (%)
Database		DSC	Precision	Sensitivity	FPR	IoU
ISBI 2015	Without augmentation	67.02	70.21	64.89	0.2	66.78
ISBI 2015	With augmentation	92.85	94.38	92.27	0.03	87.26
MICCAI 2016	Without augmentation	74.09	73.24	69.68	0.4	67.32
MICCAI 2016	With augmentation	89.14	89.76	92.14	0.05	77.22

Finally, for a more rigorous evaluation of our method, we compared it to existing segmentation approaches using the same datasets. Tables 5 and 6 summaries the results of previous studies, confirming the superiority of our method, both in terms of accuracy and robustness, for ISBI 2015 and MICCAI 2016 datasets respectively. These results underline the relevance of our methodological choices and the effectiveness of our model in segmenting MS lesions.

Table 5

Evaluation metrics for ISBI 2015 dataset. Values in bold indicate the obtained best value.
Publications	Modalities	DSC	Precision	Sensitivity
Yeeleng et al. (2020) [17]	FLAIR	60.18	42.56	55.23
Afzal et al. (2021) [18]	T1-w, T2-w, FLAIR	67	90	48
Nada et al. (2022) [28]	FLAIR	72	81	75
Sadeghibakhi et al. (2022) [31]	T1-w, FLAIR	63.21	-	-
Sarica et al. (2022) [35]	T1-w, T2-w, FLAIR	66	86	-
Hashemi et al. (2022) [30]	T2-w, FLAIR	81	84	79
Rondinella et al. (2023) [27]	FLAIR	89	91	86
Krishnan et al. (2023) [29]	T1-w, T2-w, FLAIR	67.7	86.5	-
Proposed method	FLAIR	92.85	94.38	92.27

Table 6

Evaluation metrics for MICCAI 2016 dataset. Values in bold indicate the obtained best value.
Publications	Modalities	DSC	Precision	Sensitivity	IoU
Ghosal et al. (2022) [32]	MPRAGE, FLAIR, T1-w, T1-Gd and T2/DP	76	-	65	-
Kolarik et al. (2021) [33]	FLAIR	61.1	-	60	-
Kamraoui et al. (2021) [34]	T1-w, FLAIR	63.9	76.8	60.8	-
Sarica et al. (2022) [35]	FLAIR	44.3	-	-	-
Raab et al. (2023) [36]	T1-w, T2-w, FLAIR	76.89	75.82	80.15	63.07
Kaur et al. (2024) [37]	T1-w, T2-w, FLAIR	87	84	74	25
Proposed method	FLAIR	89.14	89.76	92.14	77.22

In addition, Figs. 10(a) and 10(b) provide an in-depth analysis of the agreement between the number of actual lesions and those predicted by our model for the ISBI 2015 and MICCAI 2016 datasets, respectively. The Pearson correlation index is a key indicator of the linear relationship between predicted and actual values. In Fig. 12(a), a very high correlation with a Pearson index of 0.901 is observed for ISBI 2015. This high correlation indicates that our model is exceptionally accurate in detecting the total number of lesions, which is crucial for clinical treatment planning and patient management. Figure 12(b) also reveals a significant correlation of 0.832 for MICCAI 2016 suggesting good, if slightly poorer, model performance for potentially more varied and complex data conditions. The results obtained demonstrate not only the model's ability to accurately estimate the number of lesions, but also its potential to reduce diagnostic errors and improve the reliability of clinical analyses. This accuracy is particularly relevant in clinical environments where critical decisions are based on the detection of lesions, such as the diagnosis of neurodegenerative diseases or post-operative assessments.

MRI is a crucial tool for the segmentation of MS lesions, providing valuable assistance to experts in the diagnosis and management of patients with this disease. In this study, we developed and evaluated an automatic method for segmenting MS lesions based on a CNN inspired by the U-Net and ResNet architectures. The obtained results show robust performance, with a Dice score above 89%, a precision exceeding 89%, a sensitivity above 92% and an IoU above 75% on the ISBI 2015 and MICCAI 2016 datasets. The used architecture combines strengths of the U-Net model with the residual blocks of ResNet, to preserve information flow throughout the network and to overcome the gradient vanishing problem through shortened connections. It is distinguished by the integration of local refinement and channel weighting mechanisms, enabling more accurate detection of small lesions and better differentiation between healthy and pathological tissue. Unlike traditional approaches that rely solely on morphological or textural features, our model learns deep representations directly from the raw image data, avoiding the potential biases associated with manual feature selection. Our model is able to predict the number of lesions, even under more varied data conditions.

Despite these promising results, our method has certain limitations, notably a slight tendency to overestimate lesion volume in areas of low contrast between healthy tissue and lesions. This overestimation could be caused by over-regularization of the model, leading to excessive smoothing of lesion boundaries. In addition, more diverse data to train and test this model could be used for generalizing results to other patient populations or different MRI images acquired with distinct protocols. Although our model performed well on the available data, challenges remain particularly for lesion segmentation in images with motion artefacts and with different spatial resolutions. To overcome these limitations, ongoing efforts are needed to refine and extend this methodology. Particularly, exploring additional datasets and increasing the diversity of clinical cases could be studied. In addition, this work will be extended for experimental validation using other imaging modalities, in particular T1-w and T2-w sequences. The integration of these modalities will capture additional information on brain tissue and lesions, providing a more comprehensive view of the disease state.

Conflict of interest:

The authors declare that there are no conflicts of interest.

Ethical approval:

Not applicable.

Author Contribution

H. B.A and M.S planned and carried out the simulationsH.B.A took the lead in writing the manuscript. H. B.A, M.S and F. K. authors contributed to the final version of the manuscriptAll authors contributed to the interpretation of the results. All authors provided critical feedback and helped shape the research, analysis and manuscript.

Data Availability

Datasets used in this study, are publicly available

Lucchinetti, C. F., Parisi, J., & Bruck, W. (2005). The pathology of multiple sclerosis. Neurologic Clinics, 23(1), 77-105, vi. https://doi.org/10.1016/j.ncl.2004.09.002
Reich, D. S., Lucchinetti, C. F., & Calabresi, P. A. (2018). Multiple Sclerosis. The New England journal of medicine, 378(2), 169–180. https://doi.org/10.1056/NEJMra1401483
Zivadinov, R., Jakimovski, D., Burnham, A., Kuhle, J., Weinstock, Z., Wicks, T.R., Ramanathan, M., Sciortino, T., Ostrem, M., Suchan, C., Dwyer, M. G., Reilly, J., Bergsland, N, Schweser, F., Kennedy, C., Young-Hong, D., Eckert, S., Hojnacki, D., Benedict, R. H. B, Weinstock-Guttman, B. (2024). Neuroimaging assessment of facility-bound severely-affected MS reveals the critical role of cortical gray matter pathology: results from the CASA–MS case-controlled study. J Neurol 271, 4949–4962. https://doi.org/10.1007/s00415-024-12420-2
Genetic risk and a primary role for cell-mediated immune mechanisms in multiple sclerosis. (2011). Nature, 476(7359), 214-219. https://doi.org/10.1038/nature10251
Compston, A., & Coles, A. (2008). Multiple sclerosis. The Lancet, 372(9648), 1502-1517. https://doi.org/10.1016/S0140-6736(08)61620-7
Miller, D. H., Filippi, M., Fazekas, F., Frederiksen, J. L., Matthews, P. M., Montalban, X., & Polman, C. H. (2004). Role of magnetic resonance imaging within diagnostic criteria for multiple sclerosis. Annals of Neurology, 56(2), 273-278. https://doi.org/10.1002/ana.20156
Filippi, M., Preziosa, P., Banwell, B. L., Barkhof, F., Ciccarelli, O., De Stefano, N., Geurts, J. J. G., Paul, F., Reich, D. S., Toosy, A. T., Traboulsee, A., Wattjes, M. P., Yousry, T. A., Gass, A., Lubetzki, C., Weinshenker, B. G., & Rocca, M. A. (2019). Assessment of lesions on magnetic resonance imaging in multiple sclerosis: Practical guidelines. Brain, 142(7), 1858–1875. https://doi.org/10.1093/brain/awz144
Hemond, C. C., & Bakshi, R. (2018). Magnetic Resonance Imaging in Multiple Sclerosis. Cold Spring Harbor Perspectives in Medicine, 8(5), a028969. https://doi.org/10.1101/cshperspect.a028969
Brochet, B. (2013). L’IRM est-elle utile dans le suivi des patients atteints de sclérose en plaques? Oui. Revue Neurologique, 169(11), 858-863. https://doi.org/10.1016/j.neurol.2013.08.003
Kamraoui, R. A., Ta, V. T., Tourdias, T., Mansencal, B., Manjon, J. V., & Coupé, P. (2022). DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation. Medical Image Analysis, 76, 102312. https://doi.org/10.1016/j.media.2021.102312
Hagiwara, A., Warntjes, M., Hori, M., Andica, C., Nakazawa, M., Kumamaru, K. K., Abe, O., & Aoki, S. (2017). SyMRI of the Brain: Rapid Quantification of Relaxation Rates and Proton Density, With Synthetic MRI, Automatic Brain Segmentation, and Myelin Measurement. Investigative Radiology, 52(10), 647–657. https://doi.org/10.1097/RLI.0000000000000365
Siger, M. (2022). Magnetic Resonance Imaging in Primary Progressive Multiple Sclerosis Patients: Review. Clinical Neuroradiology, 32(3), 625–641. https://doi.org/10.1007/s00062-022-01144-3
Dachraoui, S., Mouelhi, A., Mosbeh, A., Sliti, W., Drissi, S., Solaiman, B., & Labidi, S. (2024) A machine learning approach for multiple sclerosis diagnosis through Detecron Architecture. Multimed Tools Application 83, 42837–42859. https://doi.org/10.1007/s11042-023-17055-5
Afzal, H. M. R., Luo, S., Ramadan, S., & Lechner-Scott, J. (2022). The emerging role of artificial intelligence in multiple sclerosis imaging. Multiple Sclerosis, 28(6), 849-858. https://doi.org/10.1177/1352458520966298
Bonacchi, R., Filippi, M., & Rocca, M. A. (2022). Role of artificial intelligence in MS clinical practice. NeuroImage: Clinical, 35, 103065. https://doi.org/10.1016/j.nicl.2022.103065
Jasperse, B., & Barkhof, F. (2023). Machine Learning in Multiple Sclerosis. In O. Colliot (Ed.), Machine Learning for Brain Disorders (Vol. 197, pp. 417-431). Humana. https://doi.org/10.1007/978-1-0716-3195-9_28
Vang, Y. S., Cao, Y., Chang, P. D., Chow, D. S., Brandt, A. U., Paul, F., & Xie, X. (2020). SynergyNet: A Fusion Framework for Multiple Sclerosis Brain MRI Segmentation with Local Refinement. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) (pp. 131-135). IEEE. https://doi.org/10.1109/ISBI45749.2020.9098610
Afzal, H., Luo, S., Ramadan, S., Lechner-Scott, J., Amin, M. R., Li, J., & Kamran, A. (2021). Automatic and robust segmentation of multiple sclerosis lesions with convolutional neural networks. CMC-COMPUTERS, Materials & Continua, 66(1), 977–991. https://doi.org/10.32604/cmc.2020.012448
Farabi Maleki, S., Yousefi, M., Afshar, S., Pedrammehr, S., Lim, C. P., Jafarizadeh, A., & Asadi, H. (2024). Artificial Intelligence for Multiple Sclerosis Management Using Retinal Images: Pearl, Peaks, and Pitfalls. Seminars in Ophthalmology, 39(4), 271-288. https://doi.org/10.1080/08820538.2023.2293030
Brosch, T., Tang, L. Y. W., Yoo, Y., Li, D. K. B., Traboulsee, A., & Tam, R. (2016). Deep 3D Convolutional Encoder Networks with Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation. IEEE Transactions on Medical Imaging, 35(5), 1229-1239. https://doi.org/10.1109/TMI.2016.2528821
Anastasi, G., Franchini, M., Pieroni, S., Buzzi, M., Buzzi, MC., Leporini, B., & Molinaro, S., (2024). Machine learning techniques in breast cancer preventive diagnosis: a review. Multimed Tools Appl. https://doi.org/10.1007/s11042-024-18775-y
Al-Antari, M. A. (2023). Artificial Intelligence for Medical Diagnostics-Existing and Future AI Technology. Diagnostics (Basel, Switzerland), 13(4), 688. https://doi.org/10.3390/diagnostics13040688
Zhang, B., Shi, H., & Wang, H. (2023). Machine Learning and AI in Cancer Prognosis, Prediction, and Treatment Selection: A Critical Approach. Journal of Multidisciplinary Healthcare, 16, 1779–1791. https://doi.org/10.2147/JMDH.S410301
Zhang, W., Li, Y., Ren, W., & Liu, B. (2023). Artificial intelligence technology in Alzheimer's disease research. Intractable & Rare Diseases Research, 12(4), 208–212. https://doi.org/10.5582/irdr.2023.01091
Yan, Y., Zhang, J. W., Zang, G. Y., & Pu, J. (2019). The primary use of artificial intelligence in cardiovascular diseases: What kind of potential role does artificial intelligence play in future medicine? Journal of Geriatric Cardiology, 16(8), 585–591. https://doi.org/10.11909/j.issn.1671-5411.2019.08.010
Fenneteau, A., Bourdon, P., Helbert, D., Fernandez-Maloigne, C., Habas, C., & Guillevin, R. (2021). Investigating efficient CNN architecture for multiple sclerosis lesion segmentation. Journal of Medical Imaging, 8(1), 014504. https://doi.org/10.1117/1.JMI.8.1.014504
Rondinella, A., Crispino, E., Guarnera, F., Giudice, O., Ortis, A., Russo, G., Lorenzo, C., Maimone, D., Pappalardo, F., & Battiato, S. (2023). Boosting multiple sclerosis lesion segmentation through attention mechanism. Computers in Biology and Medicine, 162, 107503. https://doi.org/10.1016/j.compbiomed.2023.107503
Messaoud, N. H., Mansour, A., Aissi, M., Ayari, R., Frih, M., Ben Abdallah, A., & Bedoui, M. H. (2022). Automated segmentation of multiple sclerosis lesions based on convolutional neural networks. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization. https://doi.org/10.1080/21681163.2022.2129454
Krishnan, A. P., Song, Z., Clayton, D., Jia, X., de Crespigny, A., & Carano, R. A. D. (2023). Multi-arm U-Net with dense input and skip connectivity for T2 lesion segmentation in clinical trials of multiple sclerosis. Scientific Reports, 13(1), 4102. https://doi.org/10.1038/s41598-023-31207-5
Hashemi, M., Akhbari, M., & Jutten, C. (2022). Delve into multiple sclerosis (MS) lesion exploration: A modified attention U-Net for MS lesion segmentation in brain MRI. Computers in Biology and Medicine, 145, 105402. https://doi.org/10.1016/j.compbiomed.2022.105402
Sadeghibakhi, M., Pourreza, H., & Mahyar, H. (2022). Multiple sclerosis lesions segmentation using attention-based CNNs in FLAIR images. IEEE Journal of Translational Engineering in Health and Medicine, 10, 1800411. https://doi.org/10.1109/JTEHM.2022.3172025
Ghosal, P., Prasad, P., & Nandi, D. (2020). A light-weighted deep learning framework for multiple sclerosis lesion segmentation. In 2020 International Conference on Image Information Processing (ICIIP). https://doi.org/10.1109/ICIIP47207.2019.8985674
Kolarik, M., Burget, R., Travieso, C., & Kocica, J. (2021). Planar 3D transfer learning for end to end unimodal MRI unbalanced data segmentation. In 2020 25th International Conference on Pattern Recognition (ICPR) (pp. 6051–6058). https://doi.org/10.1109/ICPR48806.2021.9412150
Kamraoui, R., Ta, V.-T., Tourdias, T., Mansencal, B., & Coupé, P. (2021). DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation. Medical Image Analysis, 76, 102312. https://doi.org/10.1016/j.media.2021.102312
Sarica, B., & Seker, D. (2022). New MS lesion segmentation with deep residual attention gate U-Net utilizing 2D slices of 3D MR images. Frontiers in Neuroscience, 16, 912000. https://doi.org/10.3389/fnins.2022.912000
Raab, F., Wein, S., Greenlee, M., Malloni, W., & Lang, E. (2023). A multimodal 2D convolutional neural network for multiple sclerosis lesion detection. Authorea Preprints. https://doi.org/10.36227/techrxiv.20257716.v1
Kaur, A., Kaur, L., & Singh, A. (2024). DeepCONN: Patch-wise deep convolutional neural networks for the segmentation of multiple sclerosis brain lesions. Multimedia Tools and Applications, 83(8), 24401-24433. https://doi.org/10.1007/s11042-023- 16292-y
Carass, A., Roy, S., Jog, A., Cuzzocreo, J. L., Magrath, E., Gherman, A., & Pham, D. L. (2017). Longitudinal multiple sclerosis lesion segmentation: Resource and challenge. NeuroImage, 148, 77-102. https://doi.org/10.1016/j.neuroimage.2016.12.064
Commowick, O., Kain, M., Casey, R., Ameli, R., Ferré, J. C., Kerbrat, A., & Cotton, F. (2021). Multiple sclerosis lesions segmentation from multiple experts: The MICCAI 2016 challenge dataset. Neuroimage, 244, 118589. https://doi.org/10.1016/j.neuroimage.2021.118589
Lesjak, Ž., Pernuš, F., Likar, B., & Špiclin, Ž. (2016). Validation of White-Matter Lesion Change Detection Methods on a Novel Publicly Available MRI Image Database. Neuroinform 14, 403–420. https://doi.org/10.1007/s12021-016-9301-1
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III (pp. 234-241). Springer International Publishing. https://doi.org/10.1007/978-3-319-24574-4_28
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2818-2826). https://doi.org/10.1109/CVPR.2016.308
Asgari Taghanaki, S., Zheng, Y., Zhou, S. K., Georgescu, B., Sharma, P., Xu, D., & Hamarneh, G. (2018). Combo loss: Handling input and output imbalance in multi-organ segmentation. Computerized Medical Imaging and Graphics. https://doi.org/10.1016/j.compmedimag.2019.04.005
Diaz-Hurtado, M., Martínez-Heras, E., Solana, E., Casas-Roma, J., Llufriu, S., Kanber, B., & Prados, F. (2022). Recent advances in the longitudinal segmentation of multiple sclerosis lesions on magnetic resonance imaging: a review. Neuroradiology 64, 2103–2117. https://doi.org/10.1007/s00234-022-03019-3

No competing interests reported.

A deep learning-based approach for Multiple Sclerosis Lesion Segmentation

Status:

Version 1

Abstract

Figures

I. INTRODUCTION

II. RELATED WORKS

III. METHODOLOGY

IV. RESULTS

V. DISCUSSION AND CONCLUSION

Declarations