Are multi-contrast magnetic resonance images necessary for segmenting multiple sclerosis brains? A large cohort study based on deep learning.

BACKGROUND Magnetic resonance images with multiple contrasts or sequences are commonly used for segmenting brain tissues, including lesions, in multiple sclerosis (MS). However, acquisition of images with multiple contrasts increases the scan time and complexity of the analysis, possibly introducing factors that could compromise segmentation quality. OBJECTIVE To investigate the effect of various combinations of multi-contrast images as input on the segmented volumes of gray (GM) and white matter (WM), cerebrospinal fluid (CSF), and lesions using a deep neural network. METHODS U-net, a fully convolutional neural network was used to automatically segment GM, WM, CSF, and lesions in 1000 MS patients. The input to the network consisted of 15 combinations of FLAIR, T1-, T2-, and proton density-weighted images. The Dice similarity coefficient (DSC) was evaluated to assess the segmentation performance. For lesions, true positive rate (TPR) and false positive rate (FPR) were also evaluated. In addition, the effect of lesion size on lesion segmentation was investigated. RESULTS Highest DSC was observed for all the tissue volumes, including lesions, when the input was combination of all four image contrasts. All other input combinations that included FLAIR also provided high DSC for all tissue classes. However, the quality of lesion segmentation showed strong dependence on the input images. The DSC and TPR values for inputs with the four contrast combination and FLAIR alone were very similar, but FLAIR showed a moderately higher FPR for lesion size <100 μl. For lesions smaller than 20 μl all image combinations resulted in poor performance. The segmentation quality improved with lesion size. CONCLUSIONS Best performance for segmented tissue volumes was obtained with all four image contrasts as the input, and comparable performance was attainable with FLAIR only as the input, albeit with a moderate increase in FPR for small lesions. This implies that acquisition of only FLAIR images provides satisfactory tissue segmentation. Lesion segmentation was poor for very small lesions and improved rapidly with lesion size.

[1]  P. Narayana,et al.  A comprehensive approach to the segmentation of multichannel three-dimensional MR brain images in multiple sclerosis☆ , 2013, NeuroImage: Clinical.

[2]  D. Louis Collins,et al.  Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging , 2013, Medical Image Anal..

[3]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[4]  Ponnada A. Narayana,et al.  Segmentation and quantification of black holes in multiple sclerosis , 2006, NeuroImage.

[5]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[6]  Rita Simões,et al.  Automatic segmentation of cerebral white matter hyperintensities using only 3D FLAIR images. , 2013, Magnetic resonance imaging.

[7]  Antonios Danelakis,et al.  Survey of automated multiple sclerosis lesion segmentation techniques on magnetic resonance imaging , 2018, Comput. Medical Imaging Graph..

[8]  S. Park,et al.  Methodologic Guide for Evaluating Clinical Performance and Effect of Artificial Intelligence Technology for Medical Diagnosis and Prediction. , 2018, Radiology.

[9]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[10]  Fuqiang Gao,et al.  Automatic segmentation of white matter hyperintensities in the elderly using FLAIR images at 3T , 2010, Journal of magnetic resonance imaging : JMRI.

[11]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[12]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[13]  J. Pluim,et al.  Evaluation of a deep learning approach for the segmentation of brain tissues and white matter hyperintensities of presumed vascular origin in MRI , 2017, NeuroImage: Clinical.

[14]  Frederik Barkhof,et al.  Imaging outcome measures for progressive multiple sclerosis trials , 2017, Multiple sclerosis.

[15]  D. Maintz,et al.  Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI , 2018, European Radiology.

[16]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[17]  Ponnada A. Narayana,et al.  Regional cortical thickness in relapsing remitting multiple sclerosis: A multi-center study☆ , 2012, NeuroImage: Clinical.

[18]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[19]  R. He,et al.  Unified Approach for Multiple Sclerosis Lesion Segmentation on Brain MRI , 2006, Annals of Biomedical Engineering.

[20]  Taku Komura,et al.  Segmentation of white matter hyperintensities using convolutional neural networks with global spatial information in routine clinical brain MRI with none or mild vascular pathology , 2018, Comput. Medical Imaging Graph..

[21]  M. Battaglini,et al.  Brain MRI atrophy quantification in MS , 2017, Neurology.

[22]  Jerry S. Wolinsky,et al.  The CombiRx trial of combined therapy with interferon and glatiramer acetate in relapsing remitting MS: Design and baseline characteristics. , 2012, Multiple sclerosis and related disorders.

[23]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  April Khademi,et al.  Robust White Matter Lesion Segmentation in FLAIR MRI , 2012, IEEE Transactions on Biomedical Engineering.

[25]  Koen L. Vincken,et al.  Probabilistic segmentation of brain tissue in MR imaging , 2005, NeuroImage.

[26]  Deborah Pareto,et al.  Brain Atrophy in Multiple Sclerosis: Clinical Relevance and Technical Aspects. , 2017, Neuroimaging clinics of North America.

[27]  Daniel L. Rubin,et al.  Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions , 2017, Journal of Digital Imaging.

[28]  David G. Stork,et al.  Pattern Classification , 1973 .

[29]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[30]  Jayaram K. Udupa,et al.  New variants of a method of MRI scale standardization , 2000, IEEE Transactions on Medical Imaging.

[31]  Jean-Philippe Thiran,et al.  Automated detection of white matter and cortical lesions in early stages of multiple sclerosis , 2016, Journal of magnetic resonance imaging : JMRI.