Deep neural network with generative adversarial networks pre-training for brain tumor classification based on MR images

Abstract In this paper, a new deep learning method for tumor classification in MR images is presented. A deep neural network is first pre-trained as a discriminator in a generative adversarial network (GAN) on different datasets of MR images to extract robust features and to learn the structure of MR images in its convolutional layers. Then the fully connected layers are replaced and the whole deep network is trained as a classifier to distinguish three tumor classes. The deep neural network classifier has six layers and about 1.7 million weight parameters. Pre-training as a discriminator of a GAN together with other techniques such as data augmentations (image rotation and mirroring) and dropout prevent the network from overtraining on a relatively small dataset. This method is applied to an MRI data set consists of 3064 T1-CE MR images from 233 patients, 13 images from each patient on average, with three different brain tumor types: meningioma (708 images), glioma (1426 images), and pituitary tumor (930 images). 5-Fold cross-validation is used to evaluate the performance of overall design, achieving the highest accuracy as compared to state-of-art methods.

[1]  Daniel Fabbri,et al.  Deep learning for brain tumor classification , 2017, Medical Imaging.

[2]  Konstantinos N. Plataniotis,et al.  Capsule Networks for Brain Tumor Classification Based on MRI Images and Coarse Tumor Boundaries , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Jin Liu,et al.  Applications of deep learning to MRI images: A survey , 2018, Big Data Min. Anal..

[4]  Moosa Ayati,et al.  Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms , 2019, Biocybernetics and Biomedical Engineering.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Abhinav Dhall,et al.  Dense and Diverse Capsule Networks: Making the Capsules Learn Better , 2018, ArXiv.

[7]  Hedieh Sajedi,et al.  Brain Tumor Classification via Convolutional Neural Network and Extreme Learning Machines , 2018, 2018 8th International Conference on Computer and Knowledge Engineering (ICCKE).

[8]  Arvid Lundervold,et al.  An overview of deep learning in medical imaging focusing on MRI , 2018, Zeitschrift fur medizinische Physik.

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Daniel L. Rubin,et al.  Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions , 2017, Journal of Digital Imaging.

[11]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[12]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[13]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[14]  Jiebo Luo,et al.  Image Captioning with Semantic Attention , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Ikhlas Abdel-Qader,et al.  Brain Tumor Classification via Statistical Features and Back-Propagation Neural Network , 2018, 2018 IEEE International Conference on Electro/Information Technology (EIT).

[16]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[17]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[18]  John G. Csernansky,et al.  Open Access Series of Imaging Studies: Longitudinal MRI Data in Nondemented and Demented Older Adults , 2010, Journal of Cognitive Neuroscience.

[19]  Konstantinos N. Plataniotis,et al.  Brain Tumor Type Classification via Capsule Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[20]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[21]  Tati L. R. Mengko,et al.  Brain Tumor Classification Using Convolutional Neural Network , 2018, IFMBE Proceedings.

[22]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[23]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[24]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[27]  G. Reifenberger,et al.  The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary , 2016, Acta Neuropathologica.

[28]  Zahid Mehmood,et al.  Feature enhancement framework for brain tumor segmentation and classification , 2019, Microscopy research and technique.

[29]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Kenji Suzuki,et al.  Overview of deep learning in medical imaging , 2017, Radiological Physics and Technology.

[32]  Kai Xu,et al.  Holistic Brain Tumor Screening and Classification Based on DenseNet and Recurrent Neural Network , 2018, BrainLes@MICCAI.

[33]  Nir Shavit,et al.  Generative Compression , 2017, 2018 Picture Coding Symposium (PCS).

[34]  Sebastian Nowozin,et al.  Stabilizing Training of Generative Adversarial Networks through Regularization , 2017, NIPS.

[35]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[36]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[37]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[38]  Qianjin Feng,et al.  Enhanced Performance of Brain Tumor Classification via Tumor Region Augmentation and Partition , 2015, PloS one.

[39]  Ruwan D. Nawarathna,et al.  A sophisticated convolutional neural network model for brain tumor classification , 2017, 2017 IEEE International Conference on Industrial and Information Systems (ICIIS).