Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach

ABSTRACT In this paper, we present a novel automated method for White Matter (WM) lesion segmentation of Multiple Sclerosis (MS) patient images. Our approach is based on a cascade of two 3D patch‐wise convolutional neural networks (CNN). The first network is trained to be more sensitive revealing possible candidate lesion voxels while the second network is trained to reduce the number of misclassified voxels coming from the first network. This cascaded CNN architecture tends to learn well from a small (Symbol) set of labeled data of the same MRI contrast, which can be very interesting in practice, given the difficulty to obtain manual label annotations and the large amount of available unlabeled Magnetic Resonance Imaging (MRI) data. We evaluate the accuracy of the proposed method on the public MS lesion segmentation challenge MICCAI2008 dataset, comparing it with respect to other state‐of‐the‐art MS lesion segmentation tools. Furthermore, the proposed method is also evaluated on two private MS clinical datasets, where the performance of our method is also compared with different recent public available state‐of‐the‐art MS lesion segmentation methods. At the time of writing this paper, our method is the best ranked approach on the MICCAI2008 challenge, outperforming the rest of 60 participant methods when using all the available input modalities (T1‐w, T2‐w and FLAIR), while still in the top‐rank (3rd position) when using only T1‐w and FLAIR modalities. On clinical MS data, our approach exhibits a significant increase in the accuracy segmenting of WM lesions when compared with the rest of evaluated methods, highly correlating (Symbol) also with the expected lesion volume. Symbol. No caption available. Symbol. No caption available. HIGHLIGHTSWe propose an automated WM lesion segmentation method for MS patient images.The approach relies on a cascade of two 7‐layer convolutional neural networks.We evaluate its accuracy with both the MICCAI2008 challenge and clinical MS data.Our approach is currently the best ranked method of the challenge (1th pos / 60).On MS data, the accuracy is significantly better that state‐of‐the‐art methods.

[1]  Mohammad Havaei,et al.  HeMIS: Hetero-Modal Image Segmentation , 2016, MICCAI.

[2]  Alex Chase,et al.  Parkinson disease: Facilitating detection of prodromal Parkinson disease in primary care clinics , 2015, Nature Reviews Neurology.

[3]  Thomas Brox,et al.  White Matter MS-Lesion Segmentation Using a Geometric Brain Model , 2016, IEEE Transactions on Medical Imaging.

[4]  W. L. Benedict,et al.  Multiple Sclerosis , 2007, Journal - Michigan State Medical Society.

[5]  F. Barkhof,et al.  Accurate white matter lesion segmentation by k nearest neighbor classification with tissue type priors (kNN-TTPs)☆ , 2013, NeuroImage: Clinical.

[6]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[7]  Xavier Lladó,et al.  Quantifying brain tissue volume in multiple sclerosis with automated lesion segmentation and filling , 2015, NeuroImage: Clinical.

[8]  Stephen M. Smith,et al.  Accurate, Robust, and Automated Longitudinal and Cross-Sectional Brain Change Analysis , 2002, NeuroImage.

[9]  B. Ginneken,et al.  3D Segmentation in the Clinic: A Grand Challenge , 2007 .

[10]  Alex Rovira,et al.  Segmentation of multiple sclerosis lesions in brain MRI: A review of automated approaches , 2012, Inf. Sci..

[11]  A. Oliver,et al.  Improved Automatic Detection of New T2 Lesions in Multiple Sclerosis Using Deformation Fields , 2016, American Journal of Neuroradiology.

[12]  Hao Chen,et al.  VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation , 2016, ArXiv.

[13]  Olivier Clatz,et al.  Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images , 2011, NeuroImage.

[14]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[15]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[16]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[17]  Jeffrey A. Cohen,et al.  Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria , 2011, Annals of neurology.

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  D. Louis Collins,et al.  Probabilistic Multiple Sclerosis Lesion Classification Based on Modeling Regional Intensity Variability and Local Neighborhood Information , 2015, IEEE Transactions on Biomedical Engineering.

[20]  Ludovica Griffanti,et al.  BIANCA (Brain Intensity AbNormality Classification Algorithm): A new tool for automated segmentation of white matter hyperintensities , 2016, NeuroImage.

[21]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[22]  D. Louis Collins,et al.  Rotation-invariant multi-contrast non-local means for MS lesion segmentation , 2015, NeuroImage: Clinical.

[23]  T. Arbel,et al.  HIERARCHICAL MRF AND RANDOM FOREST SEGMENTATION OF MS LESIONS AND HEALTHY TISSUES IN BRAIN MRI , 2015 .

[24]  Bostjan Likar,et al.  Combining Unsupervised and Supervised Methods for Lesion Segmentation , 2015, Brainles@MICCAI.

[25]  Bernhard Hemmer,et al.  An automated tool for detection of FLAIR-hyperintense white-matter lesions in Multiple Sclerosis , 2012, NeuroImage.

[26]  F. Barkhof,et al.  Evidence-based guidelines: MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis—clinical implementation in the diagnostic process , 2015, Nature Reviews Neurology.

[27]  Razvan Pascanu,et al.  Theano: Deep Learning on GPUs with Python , 2012 .

[28]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[29]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[30]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[31]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[32]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[33]  Yaozong Gao,et al.  LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images , 2014, NeuroImage.

[34]  A. Oliver,et al.  A toolbox for multiple sclerosis lesion segmentation , 2015, Neuroradiology.

[35]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[36]  Paul M. Thompson,et al.  Intensity non-uniformity correction using N3 on 3-T scanners with multichannel phased array coils , 2008, NeuroImage.

[37]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[38]  Max A. Viergever,et al.  Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities , 2016, MICCAI.

[39]  Simon K. Warfield,et al.  A Model of Population and Subject (MOPS) Intensities With Application to Multiple Sclerosis Lesion Segmentation , 2015, IEEE Transactions on Medical Imaging.

[40]  Olivier Commowick,et al.  MSSEG Challenge Proceedings: Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure , 2016, MICCAI 2016.

[41]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  A. Newberg,et al.  Neuroimaging in traumatic brain imaging , 2005, NeuroRX.

[43]  Arnau Oliver,et al.  Automated Detection of Lupus White Matter Lesions in MRI , 2016, Front. Neuroinform..

[44]  Jean-Philippe Thiran,et al.  Automated detection of white matter and cortical lesions in early stages of multiple sclerosis , 2016, Journal of magnetic resonance imaging : JMRI.

[45]  M. Calabrese,et al.  Effect of disease-modifying drugs on cortical lesions and atrophy in relapsing–remitting multiple sclerosis , 2012, Multiple sclerosis.

[46]  Michael W. L. Chee,et al.  Improvement of brain segmentation accuracy by optimizing non-uniformity correction using N3 , 2009, NeuroImage.

[47]  Alan C. Evans,et al.  A nonparametric method for automatic correction of intensity nonuniformity in MRI data , 1998, IEEE Transactions on Medical Imaging.

[48]  Arnau Oliver,et al.  BOOST: A supervised approach for multiple sclerosis lesion segmentation , 2014, Journal of Neuroscience Methods.

[49]  Max A. Viergever,et al.  Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks , 2016, Medical Image Anal..

[50]  Max A. Viergever,et al.  Automatic segmentation of MR brain images of preterm infants using supervised classification , 2015, NeuroImage.

[51]  A. Oliver,et al.  A white matter lesion-filling approach to improve brain tissue volume measurements , 2014, NeuroImage: Clinical.

[52]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[53]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[54]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[55]  L Steinman,et al.  Multiple Sclerosis: A Coordinated Immunological Attack against Myelin in the Central Nervous System , 1996, Cell.

[56]  O. Ciccarelli,et al.  MRI CRITERIA FOR THE DIAGNOSIS OF MULTIPLE SCLEROSIS: MAGNIMS CONSENSUS GUIDELINES , 2016, The Lancet Neurology.

[57]  Alexis Roche,et al.  Automated Detection of White-matter and Cortical Lesions in MP2RAGE at Ultra-High Field using a Single Scan , 2017 .

[58]  D. Louis Collins,et al.  Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging , 2013, Medical Image Anal..

[59]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[60]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[61]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[62]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.