Multiple Sclerosis Lesion Synthesis in MRI Using an Encoder-Decoder U-NET

Magnetic resonance imaging (MRI) synthesis has attracted attention due to its various applications in the medical imaging domain. In this paper, we propose generating synthetic multiple sclerosis (MS) lesions on MRI images with the final aim to improve the performance of supervised machine learning algorithms, therefore, avoiding the problem of the lack of available ground truth. We propose a two-input two-output fully convolutional neural network model for MS lesion synthesis in MRI images. The lesion information is encoded as discrete binary intensity level masks passed to the model and stacked with the input images. The model is trained end-to-end without the need for manually annotating the lesions in the training set. We then perform the generation of synthetic lesions on healthy images via registration of patient images, which are subsequently used for data augmentation to increase the performance for supervised MS lesion detection algorithms. Our pipeline is evaluated on MS patient data from an in-house clinical dataset and the public ISBI2015 challenge dataset. The evaluation is based on measuring the similarities between the real and the synthetic images as well as in terms of lesion detection performance by segmenting both the original and synthetic images individually using a state-of-the-art segmentation framework. We also demonstrate the usage of synthetic MS lesions generated on healthy images as data augmentation. We analyze a scenario of limited training data (one-image training) to demonstrate the effect of the data augmentation on both datasets. Our results significantly show the effectiveness of the usage of synthetic MS lesion images. For the ISBI2015 challenge, our one-image model trained using only a single image plus the synthetic data augmentation strategy showed a performance similar to that of other CNN methods that were fully trained using the entire training set, yielding a comparable human expert rater performance.

[1]  Sébastien Ourselin,et al.  Bayesian Model Selection for Pathological Neuroimaging Data Applied to White Matter Lesion Segmentation , 2015, IEEE Transactions on Medical Imaging.

[2]  O. Ciccarelli,et al.  MRI CRITERIA FOR THE DIAGNOSIS OF MULTIPLE SCLEROSIS: MAGNIMS CONSENSUS GUIDELINES , 2016, The Lancet Neurology.

[3]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[4]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[5]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Saurabh Jain,et al.  Automatic segmentation and volumetry of multiple sclerosis brain lesions from MR images , 2015, NeuroImage: Clinical.

[8]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[9]  Simon K. Warfield,et al.  Asymmetric Loss Functions and Deep Densely-Connected Networks for Highly-Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection , 2018, IEEE Access.

[10]  Sotirios A. Tsaftaris,et al.  Multimodal MR Synthesis via Modality-Invariant Latent Representation , 2018, IEEE Transactions on Medical Imaging.

[11]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[12]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[13]  Joaquim Salvi,et al.  A supervised framework with intensity subtraction and deformation field features for the detection of new T2-w lesions in multiple sclerosis , 2017, NeuroImage: Clinical.

[14]  Daniel Rueckert,et al.  Pseudo-healthy Image Synthesis for White Matter Lesion Segmentation , 2016, SASHIMI@MICCAI.

[15]  Alex Rovira,et al.  Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach , 2017, NeuroImage.

[16]  John Muschelli,et al.  MIMoSA: An Automated Method for Intermodal Segmentation Analysis of Multiple Sclerosis Brain Lesions , 2018, Journal of neuroimaging : official journal of the American Society of Neuroimaging.

[17]  Simon Andermatt,et al.  Automated Segmentation of Multiple Sclerosis Lesions Using Multi-dimensional Gated Recurrent Units , 2017, BrainLes@MICCAI.

[18]  Paul M. Thompson,et al.  Robust Brain Extraction Across Datasets and Comparison With Publicly Available Methods , 2011, IEEE Transactions on Medical Imaging.

[19]  F. Barkhof,et al.  Evidence-based guidelines: MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis—clinical implementation in the diagnostic process , 2015, Nature Reviews Neurology.

[20]  Alex Rovira,et al.  Segmentation of multiple sclerosis lesions in brain MRI: A review of automated approaches , 2012, Inf. Sci..

[21]  Peter A. Calabresi,et al.  A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions , 2010, NeuroImage.

[22]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[23]  Snehashis Roy,et al.  Longitudinal multiple sclerosis lesion segmentation: Resource and challenge , 2017, NeuroImage.

[24]  Marleen de Bruijne,et al.  Why Does Synthesized Data Improve Multi-sequence Classification? , 2015, MICCAI.

[25]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[26]  Jung-Hwan Oh,et al.  Real Data Augmentation for Medical Image Classification , 2017, CVII-STENT/LABELS@MICCAI.

[27]  Sébastien Ourselin,et al.  Global image registration using a symmetric block-matching approach , 2014, Journal of medical imaging.

[28]  B. Ginneken,et al.  3D Segmentation in the Clinic: A Grand Challenge , 2007 .

[29]  M. Battaglini,et al.  Evaluating and reducing the impact of white matter lesions on brain volume measurements , 2012, Human brain mapping.

[30]  Sébastien Ourselin,et al.  Fast free-form deformation using graphics processing units , 2010, Comput. Methods Programs Biomed..

[31]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[32]  D. Rueckert,et al.  White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks , 2017, NeuroImage: Clinical.

[33]  Ben Glocker,et al.  Is Synthesizing MRI Contrast Useful for Inter-modality Analysis? , 2013, MICCAI.

[34]  Alex Rovira,et al.  Automatic multiple sclerosis lesion detection in brain MRI by FLAIR thresholding , 2014, Comput. Methods Programs Biomed..

[35]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[36]  Deniz Erdogmus,et al.  Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks , 2017, MLMI@MICCAI.

[37]  Christian Barillot,et al.  Classification of multiple sclerosis lesions using adaptive dictionary learning , 2015, Comput. Medical Imaging Graph..

[38]  Hayit Greenspan,et al.  Multi-view longitudinal CNN for multiple sclerosis lesion segmentation , 2017, Eng. Appl. Artif. Intell..