Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation

We propose a novel segmentation approach based on deep 3D convolutional encoder networks with shortcut connections and apply it to the segmentation of multiple sclerosis (MS) lesions in magnetic resonance images. Our model is a neural network that consists of two interconnected pathways, a convolutional pathway, which learns increasingly more abstract and higher-level image features, and a deconvolutional pathway, which predicts the final segmentation at the voxel level. The joint training of the feature extraction and prediction pathways allows for the automatic learning of features at different scales that are optimized for accuracy for any given combination of image types and segmentation task. In addition, shortcut connections between the two pathways allow high- and low-level features to be integrated, which enables the segmentation of lesions across a wide range of sizes. We have evaluated our method on two publicly available data sets (MICCAI 2008 and ISBI 2015 challenges) with the results showing that our method performs comparably to the top-ranked state-of-the-art methods, even when only relatively small data sets are available for training. In addition, we have compared our method with five freely available and widely used MS lesion segmentation methods (EMS, LST-LPA, LST-LGA, Lesion-TOADS, and SLS) on a large data set from an MS clinical trial. The results show that our method consistently outperforms these other methods across a wide range of lesion sizes.

[1]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[2]  Sepp Hochreiter,et al.  Untersuchungen zu dynamischen neuronalen Netzen , 1991 .

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  Koenraad Van Leemput,et al.  Automated segmentation of multiple sclerosis lesions by model outlier detection , 2001, IEEE Transactions on Medical Imaging.

[5]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[6]  L. Kappos,et al.  Long-term subcutaneous interferon beta-1a therapy in patients with relapsing-remitting MS , 2006, Neurology.

[7]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[8]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[9]  B. Ginneken,et al.  3D Segmentation in the Clinic: A Grand Challenge , 2007 .

[10]  A. Traboulsee,et al.  Reduction in magnetic resonance imaging T2 burden of disease in patients with relapsing-remitting multiple sclerosis: analysis of 48-week data from the EVIDENCE (EVidence of Interferon Dose-response: European North American Comparative Efficacy) study , 2008, BMC neurology.

[11]  D. Collins,et al.  MS Lesion Segmentation using Markov Random Fields , 2009 .

[12]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[13]  Olivier Clatz,et al.  Spatial Decision Forests for MS Lesion Segmentation in Multi-Channel MR Images , 2010, MICCAI.

[14]  Peter A. Calabresi,et al.  A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions , 2010, NeuroImage.

[15]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[16]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[17]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[18]  Clément Farabet,et al.  Torch7: A Matlab-like Environment for Machine Learning , 2011, NIPS 2011.

[19]  D. Louis Collins,et al.  Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation , 2011, NeuroImage.

[20]  Graham W. Taylor,et al.  Adaptive deconvolutional networks for mid and high level feature learning , 2011, 2011 International Conference on Computer Vision.

[21]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[22]  Razvan Pascanu,et al.  Theano: new features and speed improvements , 2012, ArXiv.

[23]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Bernhard Hemmer,et al.  An automated tool for detection of FLAIR-hyperintense white-matter lesions in Multiple Sclerosis , 2012, NeuroImage.

[26]  Daniel Rueckert,et al.  Multiple Sclerosis Lesion Segmentation Using Dictionary Learning and Sparse Coding , 2013, MICCAI.

[27]  D. Louis Collins,et al.  Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging , 2013, Medical Image Anal..

[28]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[29]  John Tran,et al.  cuDNN: Efficient Primitives for Deep Learning , 2014, ArXiv.

[30]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[31]  Xiaogang Wang,et al.  Fully Convolutional Neural Networks for Crowd Segmentation , 2014, ArXiv.

[32]  Youngjin Yoo,et al.  Deep Learning of Image Features from Unlabeled Data for Multiple Sclerosis Lesion Segmentation , 2014, MLMI.

[33]  D. Louis Collins,et al.  Rotation-invariant multi-contrast non-local means for MS lesion segmentation , 2015, NeuroImage: Clinical.

[34]  Harm de Vries,et al.  RMSProp and equilibrated adaptive learning rates for non-convex optimization. , 2015 .

[35]  Doina Precup,et al.  IMaGe: Iterative Multilevel Probabilistic Graphical Model for Detection and Segmentation of Multiple Sclerosis Lesions in Brain MRI , 2015, IPMI.

[36]  T. Arbel,et al.  HIERARCHICAL MRF AND RANDOM FOREST SEGMENTATION OF MS LESIONS AND HEALTHY TISSUES IN BRAIN MRI , 2015 .

[37]  Roger C. Tam,et al.  Efficient Training of Convolutional Deep Belief Networks in the Frequency Domain for Application to High-Resolution 2D and 3D Images , 2015, Neural Computation.

[38]  Bostjan Likar,et al.  Combining Unsupervised and Supervised Methods for Lesion Segmentation , 2015, Brainles@MICCAI.

[39]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[40]  Simon K. Warfield,et al.  A Model of Population and Subject (MOPS) Intensities With Application to Multiple Sclerosis Lesion Segmentation , 2015, IEEE Transactions on Medical Imaging.

[41]  A. Oliver,et al.  A toolbox for multiple sclerosis lesion segmentation , 2015, Neuroradiology.

[42]  Sébastien Ourselin,et al.  Bayesian Model Selection for Pathological Neuroimaging Data Applied to White Matter Lesion Segmentation , 2015, IEEE Transactions on Medical Imaging.

[43]  Trevor Darrell,et al.  Fully convolutional networks for semantic segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[45]  H. Handels.  MS-LESION SEGMENTATION IN MRI WITH RANDOM FORESTS , 2015 .

[46]  Lisa Tang,et al.  Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation , 2015, MICCAI.