Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patient's neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to extract and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

[1]  Mohsen Ghafoorian,et al.  CONVOLUITIONAL NEURAL NETWORKS FOR MS LESION SEGMENTATION , METHOD DESCRIPTION OF DIAG TEAM , 2015 .

[2]  Olivier Commowick,et al.  MSSEG Challenge Proceedings: Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure , 2016, MICCAI 2016.

[3]  Yang Gao,et al.  MS-GAN: GAN-Based Semantic Segmentation of Multiple Sclerosis Lesions in Brain Magnetic Resonance Imaging , 2018, 2018 Digital Image Computing: Techniques and Applications (DICTA).

[4]  T. Arbel,et al.  HIERARCHICAL MRF AND RANDOM FOREST SEGMENTATION OF MS LESIONS AND HEALTHY TISSUES IN BRAIN MRI , 2015 .

[5]  A. Oliver,et al.  Improved Automatic Detection of New T2 Lesions in Multiple Sclerosis Using Deformation Fields , 2016, American Journal of Neuroradiology.

[6]  Olivier Commowick,et al.  Automatic Multiple Sclerosis lesion segmentation from Intensity-Normalized multi-channel MRI , 2016 .

[7]  Yi Wang,et al.  Geometric Loss For Deep Multiple Sclerosis Lesion Segmentation , 2020, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[8]  Martin Styner,et al.  Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure , 2018, bioRxiv.

[9]  Ronald M. Summers,et al.  A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises , 2020, Proceedings of the IEEE.

[10]  Paul M. Thompson,et al.  Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods , 2011, IEEE Transactions on Medical Imaging.

[11]  Nico Karssemeijer,et al.  Deep multi-scale location-aware 3D convolutional neural networks for automated detection of lacunes of presumed vascular origin , 2016, NeuroImage: Clinical.

[12]  B. Ginneken,et al.  3D Segmentation in the Clinic: A Grand Challenge , 2007 .

[13]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[14]  George Papandreou,et al.  Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Joaquim Salvi,et al.  A fully convolutional neural network for new T2-w lesion detection in multiple sclerosis , 2019, NeuroImage: Clinical.

[17]  Doina Precup,et al.  Exploring Uncertainty Measures in Deep Networks for Multiple Sclerosis Lesion Detection and Segmentation , 2018, MICCAI.

[18]  Michael H. Barnett,et al.  Two Time Point MS Lesion Segmentation in Brain MRI: An Expectation-Maximization Framework , 2016, Front. Neurosci..

[19]  Peter A. Calabresi,et al.  Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis , 2020, Scientific Reports.

[20]  K. Selmaj,et al.  Alemtuzumab vs. interferon beta-1a in early multiple sclerosis. , 2008, The New England journal of medicine.

[21]  Maria Assunta Rocca,et al.  Multi-branch convolutional neural network for multiple sclerosis lesion segmentation , 2018, NeuroImage.

[22]  Enzo Ferrante,et al.  Unsupervised domain adaptation via CycleGAN for white matter hyperintensity segmentation in multicenter MR images , 2020, Symposium on Medical Information Processing and Analysis.

[23]  Xavier Lladó,et al.  A subtraction pipeline for automatic detection of new appearing multiple sclerosis lesions in longitudinal studies , 2014, Neuroradiology.

[24]  Mark W. Woolrich,et al.  FSL , 2012, NeuroImage.

[25]  O. Ciccarelli,et al.  2021 MAGNIMS–CMSC–NAIMS consensus recommendations on the use of MRI in patients with multiple sclerosis , 2021, The Lancet Neurology.

[26]  Grégoire Malandain,et al.  An Automatic Segmentation of T2-FLAIR Multiple Sclerosis Lesions , 2008, The MIDAS Journal.

[27]  Sudipta Roy,et al.  An effective method for computerized prediction and segmentation of multiple sclerosis lesions in brain MRI , 2017, Comput. Methods Programs Biomed..

[28]  Andreas Keller,et al.  Swarm Learning for decentralized and confidential clinical machine learning , 2021, Nature.

[29]  John Muschelli,et al.  MIMoSA: An Automated Method for Intermodal Segmentation Analysis of Multiple Sclerosis Brain Lesions , 2018, Journal of neuroimaging : official journal of the American Society of Neuroimaging.

[30]  Hayit Greenspan,et al.  Longitudinal Multiple Sclerosis Lesion Segmentation Using Multi-view Convolutional Neural Networks , 2016, LABELS/DLMIA@MICCAI.

[31]  D. Louis Collins,et al.  Multiple Sclerosis lesion segmentation using an automated multimodal Graph Cut , 2016 .

[32]  A. Kouzani,et al.  Segmentation of multiple sclerosis lesions in MR images: a review , 2011, Neuroradiology.

[33]  Chandan Chakraborty,et al.  A Fast Approach to Automatic Detection of Brain Lesions , 2016, BrainLes@MICCAI.

[34]  T. Chitnis,et al.  A roadmap to precision medicine for multiple sclerosis , 2020, Multiple sclerosis.

[35]  Nils Gessert,et al.  Multiple Sclerosis Lesion Activity Segmentation with Attention-Guided Two-Path CNNs , 2020, Comput. Medical Imaging Graph..

[36]  D. Louis Collins,et al.  Rotation-invariant multi-contrast non-local means for MS lesion segmentation , 2015, NeuroImage: Clinical.

[37]  Amol S Pednekar,et al.  Automated patient‐specific optimization of three‐dimensional double‐inversion recovery magnetic resonance imaging , 2016, Magnetic resonance in medicine.

[38]  Daniel Rueckert,et al.  Multiple Sclerosis Lesion Segmentation Using Dictionary Learning and Sparse Coding , 2013, MICCAI.

[39]  Aaron Carass,et al.  DeepHarmony: A deep learning approach to contrast harmonization across scanner changes. , 2019, Magnetic resonance imaging.

[40]  Simon K. Warfield,et al.  Asymmetric Loss Functions and Deep Densely-Connected Networks for Highly-Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection , 2018, IEEE Access.

[41]  Nico Karssemeijer,et al.  Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation , 2017, MICCAI.

[42]  C. Pozzilli,et al.  A Combined Radiomics and Machine Learning Approach to Overcome the Clinicoradiologic Paradox in Multiple Sclerosis , 2021, American Journal of Neuroradiology.

[43]  Isabelle Bloch,et al.  A 3D hierarchical multimodal detection and segmentation method for multiple sclerosis lesions in MRI , 2016 .

[44]  F. Barkhof,et al.  Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (kNN-TTPs)☆ , 2013, NeuroImage: Clinical.

[45]  Andrew Doyle,et al.  Lesion Detection, Segmentation and Prediction in Multiple Sclerosis Clinical Trials , 2017, BrainLes@MICCAI.

[46]  Sébastien Ourselin,et al.  Co-registration of Histological, Optical and MR Data of the Human Brain , 2002, MICCAI.

[47]  Olivier Clatz,et al.  Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images , 2011, NeuroImage.

[48]  Marcel Bengs,et al.  4D Deep Learning for Multiple Sclerosis Lesion Activity Segmentation , 2020, ArXiv.

[49]  Joaquim Salvi,et al.  One-shot domain adaptation in multiple sclerosis lesion segmentation using convolutional neural networks , 2018, NeuroImage: Clinical.

[50]  Olivier Commowick,et al.  Automatic graph cut segmentation of multiple sclerosis lesions , 2015 .

[51]  A. Oliver,et al.  A toolbox for multiple sclerosis lesion segmentation , 2015, Neuroradiology.

[52]  Koenraad Van Leemput,et al.  A contrast-adaptive method for simultaneous whole-brain and lesion segmentation in multiple sclerosis , 2020, NeuroImage.

[53]  Guang-Zhong Yang,et al.  Deep Learning for Health Informatics , 2017, IEEE Journal of Biomedical and Health Informatics.

[54]  D. Louis Collins,et al.  Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging , 2013, Medical Image Anal..

[55]  R. Wiest,et al.  Simultaneous lesion and brain segmentation in multiple sclerosis using deep neural networks , 2021, Scientific Reports.

[56]  Simon K. Warfield,et al.  A Model of Population and Subject (MOPS) Intensities With Application to Multiple Sclerosis Lesion Segmentation , 2015, IEEE Transactions on Medical Imaging.

[57]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[58]  Rohit Bakshi,et al.  Multiple Sclerosis Lesion Segmentation with Tiramisu and 2.5D Stacked Slices , 2019, MICCAI.

[59]  Christophe Collet,et al.  MS Lesion Segmentation based on Hidden Markov Chains , 2008, The MIDAS Journal.

[60]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[61]  Antonios Danelakis,et al.  Survey of automated multiple sclerosis lesion segmentation techniques on magnetic resonance imaging , 2018, Comput. Medical Imaging Graph..

[62]  Sotirios A. Tsaftaris,et al.  AI in Medical Imaging Informatics: Current Challenges and Future Directions , 2020, IEEE Journal of Biomedical and Health Informatics.

[63]  Alex Rovira,et al.  Segmentation of multiple sclerosis lesions in brain MRI: A review of automated approaches , 2012, Inf. Sci..

[64]  Örjan Smedby,et al.  Automatic multiple sclerosis lesion segmentation using hybrid artificial neural networks , 2016 .

[65]  Snehashis Roy,et al.  Longitudinal multiple sclerosis lesion segmentation: Resource and challenge , 2017, NeuroImage.

[66]  Simon Andermatt,et al.  Automated Segmentation of Multiple Sclerosis Lesions Using Multi-dimensional Gated Recurrent Units , 2017, BrainLes@MICCAI.

[67]  Sunanda Mitra,et al.  Automated Segmentation of MS Lesions in MR Images Based on an Information Theoretic Clustering and Contrast Transformations , 2015 .

[68]  Michel Dojat,et al.  Automatic multiple sclerosis lesion segmentation with P-LOCUS , 2016 .

[69]  Amrita Kaur,et al.  State-of-the-Art Segmentation Techniques and Future Directions for Multiple Sclerosis Brain Lesions , 2020, Archives of Computational Methods in Engineering.

[70]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[71]  Mário João Fartaria,et al.  Multiple sclerosis cortical and WM lesion segmentation at 3T MRI: a deep learning method based on FLAIR and MP2RAGE , 2020, NeuroImage: Clinical.

[72]  Nicolas Courty,et al.  Unsupervised Domain Adaptation With Optimal Transport in Multi-Site Segmentation of Multiple Sclerosis Lesions From MRI Data , 2020, Frontiers in Computational Neuroscience.

[73]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[74]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[75]  P. Bazin,et al.  Multiple Sclerosis Lesion Segmentation Using Statistical and Topological Atlases , 2008, The MIDAS Journal.

[76]  H. Handels MS-LESION SEGMENTATION IN MRI WITH RANDOM FORESTS , 2015 .

[77]  P. Villoslada,et al.  Multiple Sclerosis International Federation (MSIF) , 2011 .

[78]  W. L. Benedict,et al.  Multiple Sclerosis , 2007, Journal - Michigan State Medical Society.

[79]  Alex Rovira,et al.  Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach , 2017, NeuroImage.

[80]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[81]  Amod Jog,et al.  Multi-output decision trees for lesion segmentation in multiple sclerosis , 2015, Medical Imaging.

[82]  Tom Gundersen,et al.  Nabla-net: A Deep Dag-Like Convolutional Architecture for Biomedical Image Segmentation , 2016, BrainLes@MICCAI.

[83]  Saurabh Jain,et al.  Automatic segmentation and volumetry of multiple sclerosis brain lesions from MR images , 2015, NeuroImage: Clinical.

[84]  Nicholas Ayache,et al.  Spatially Adaptive Random Forests , 2013, 2013 IEEE 10th International Symposium on Biomedical Imaging.

[85]  Richard McKinley,et al.  Automatic detection of lesion load change in Multiple Sclerosis using convolutional neural networks with segmentation confidence , 2019, NeuroImage: Clinical.

[86]  Peter A. Calabresi,et al.  Multiple Sclerosis Lesion Segmentation from Brain MRI via Fully Convolutional Neural Networks , 2018, ArXiv.

[87]  Jens Petersen,et al.  nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation , 2020, Nature Methods.

[88]  J. Haynes,et al.  Can we overcome the ‘clinico-radiological paradox’ in multiple sclerosis? , 2012, Journal of Neurology.

[89]  Karl J. Friston,et al.  MRI and PET Coregistration—A Cross Validation of Statistical Parametric Mapping and Automated Image Registration , 1997, NeuroImage.

[90]  Sergiu Groppa,et al.  Automated segmentation of changes in FLAIR-hyperintense white matter lesions in multiple sclerosis on serial magnetic resonance imaging , 2019, NeuroImage: Clinical.

[91]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[92]  Rickmer Braren,et al.  Secure, privacy-preserving and federated machine learning in medical imaging , 2020, Nature Machine Intelligence.

[93]  Ramanathan Muthuganapathy,et al.  LONGITUDINAL MULTIPLE SCLEROSIS LESION SEGMENTATION USING 3 D CONVOLUTIONAL NEURAL NETWORKS , 2015 .

[94]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[95]  Xavier Lladó,et al.  Automated detection of multiple sclerosis lesions in serial brain MRI , 2012, Neuroradiology.

[96]  Jeffrey A. Cohen,et al.  Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria , 2011, Annals of neurology.

[97]  Bernhard Hemmer,et al.  An automated tool for detection of FLAIR-hyperintense white-matter lesions in Multiple Sclerosis , 2012, NeuroImage.

[98]  Hao Guan,et al.  Domain Adaptation for Medical Image Analysis: A Survey , 2021, IEEE Transactions on Biomedical Engineering.