Brain MRI Tumor Segmentation with Adversarial Networks

Deep Learning is a promising approach to either automate or simplify several tasks in the healthcare domain. In this work, we introduce SegAN-CAT, an end-to-end approach to brain tumor segmentation in Magnetic Resonance Images (MRI), based on Adversarial Networks. In particular, we extend SegAN, successfully applied to the same task in a previous work, in two respects: (i) we used a different model input and (ii) we employed a modified loss function to train the model. We tested our approach on two large datasets, made available by the Brain Tumor Image Segmentation Benchmark (BraTS). First, we trained and tested some segmentation models assuming the availability of all the major MRI contrast modalities, i.e., T1-weighted, T1 weighted contrast enhanced, T2-weighted, and T2-FLAIR. However, as these four modalities are not always all available for each patient, we also trained and tested four segmentation models that take as input MRIs acquired with a single contrast modality. Finally, we proposed to apply transfer learning across different contrast modalities to improve the performance of these single-modality models. Our results are promising and show that not only SegAN-CAT is able to outperform SegAN when all the four modalities are available, but also that transfer learning can actually lead to better performances when only a single modality is available.

[1]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[2]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[3]  Lei Wang,et al.  3D cGAN based cross-modality MR image synthesis for brain tumor segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[4]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[5]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[6]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[7]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[8]  Mitko Veta,et al.  Adversarial Training and Dilated Convolutions for Brain MRI Segmentation , 2017, DLMIA/ML-CDS@MICCAI.

[9]  Joseph O. Deasy,et al.  Tumor-Aware, Adversarial Domain Adaptation from CT to MRI for Lung Cancer Segmentation , 2018, MICCAI.

[10]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[11]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[12]  Carl Boettiger,et al.  An introduction to Docker for reproducible research , 2014, OPSR.

[13]  Max A. Viergever,et al.  Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities , 2016, MICCAI.

[14]  Irene Y. H. Gu,et al.  Cross-Modality Augmentation of Brain Mr Images Using a Novel Pairwise Generative Adversarial Network for Enhanced Glioma Classification , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[15]  Y. Yao,et al.  On Early Stopping in Gradient Descent Learning , 2007 .

[16]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[17]  Masoom A. Haider,et al.  A Transfer Learning Approach for Automated Segmentation of Prostate Whole Gland and Transition Zone in Diffusion Weighted MRI , 2019, ArXiv.

[18]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[19]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[20]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[21]  Stephen V. Stehman,et al.  Selecting and interpreting measures of thematic classification accuracy , 1997 .

[22]  Robert M. Haralick,et al.  Feature normalization and likelihood-based similarity measures for image retrieval , 2001, Pattern Recognit. Lett..

[23]  Yaël Frégier,et al.  Geometric Science of Information: 5th International Conference, GSI 2021, Paris, France, July 21–23, 2021, Proceedings , 2019, GSI.

[24]  Stephen Gould,et al.  Decomposing a scene into geometric and semantically consistent regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[25]  Bogdan Raducanu,et al.  Transferring GANs: generating images from limited data , 2018, ECCV.

[26]  Tolga Çukur,et al.  A Transfer‐Learning Approach for Accelerated MRI Using Deep Neural Networks , 2017, Magnetic resonance in medicine.

[27]  Aykut Erdem,et al.  Image Synthesis in Multi-Contrast MRI With Conditional Generative Adversarial Networks , 2018, IEEE Transactions on Medical Imaging.

[28]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[29]  Camille Couprie,et al.  Semantic Segmentation using Adversarial Networks , 2016, NIPS 2016.

[30]  Parashkev Nachev,et al.  PIMMS: Permutation Invariant Multi-Modal Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[31]  Marc Modat,et al.  Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation , 2019, MICCAI.

[32]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[33]  Nico Karssemeijer,et al.  Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation , 2017, MICCAI.

[34]  M. Jorge Cardoso,et al.  Simultaneous synthesis of FLAIR and segmentation of white matter hypointensities from T1 MRIs , 2018, ArXiv.

[35]  Ghassan Hamarneh,et al.  Missing MRI Pulse Sequence Synthesis Using Multi-Modal Generative Adversarial Network , 2019, IEEE Transactions on Medical Imaging.

[36]  Mohammad Havaei,et al.  HeMIS: Hetero-Modal Image Segmentation , 2016, MICCAI.

[37]  Isabelle Bloch,et al.  From neonatal to adult brain MR image segmentation in a few seconds using 3D-like fully convolutional network and transfer learning , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[38]  Dirk Merkel,et al.  Docker: lightweight Linux containers for consistent development and deployment , 2014 .

[39]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[40]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[41]  Jan Kautz,et al.  High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[43]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[44]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..