Using generative adversarial networks and transfer learning for breast cancer detection by convolutional neural networks

In the U.S., breast cancer is diagnosed in about 12% of women during their lifetime and it is the second leading reason for women’s death. Since early diagnosis could improve treatment outcomes and longer survival times for breast cancer patients, it is significant to develop breast cancer detection techniques. The Convolutional Neural Network (CNN) can extract features from images automatically and then perform classification. To train the CNN from scratch, however, requires a large number of labeled images, which is infeasible for some kinds of medical image data such as mammographic tumor images. In this paper, we proposed two solutions to the lack of training images. 1)To generate synthetic mammographic images for training by the Generative Adversarial Network (GAN). Adding GAN generated images made to train CNN from scratch successful and adding more GAN images improved CNN’s validation accuracy to at most (best) 98.85%. 2)To apply transfer learning in CNN. We used the pre-trained VGG-16 model to extract features from input mammograms and used these features to train a Neural Network (NN)-classifier. The stable average validation accuracy converged at about 91.48% for classifying abnormal vs. normal cases in the DDSM database. Then, we combined the two deep-learning based technologies together. That is to apply GAN for image augmentation and transfer learning in CNN for breast cancer detection. To the training set including real and GAN augmented images, although transfer learning model did not perform better than the CNN, the speed of training transfer learning model was about 10 times faster than CNN training. Adding GAN images can help training avoid over-fitting and image augmentation by GAN is necessary to train CNN classifiers from scratch. On the other hand, transfer learning is necessary to be applied for training on pure real images. To apply GAN to augment training images for training CNN classifier obtained the best classification performance.

[1]  Ulas Bagci,et al.  How to fool radiologists with generative adversarial networks? A visual turing test for lung cancer diagnosis , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[2]  Klaus H. Maier-Hein,et al.  Adversarial Networks for the Detection of Aggressive Prostate Cancer , 2017, ArXiv.

[3]  Pascal Vincent,et al.  The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training , 2009, AISTATS.

[4]  Aijun Liu,et al.  Abnormal Breast Detection in Mammogram Images by Feed-forward Neural Network Trained by Jaya Algorithm , 2017, Fundam. Informaticae.

[5]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[6]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[7]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[9]  Gustavo Carneiro,et al.  The Automated Learning of Deep Features for Breast Mass Classification from Mammograms , 2016, MICCAI.

[10]  Atsuto Maki,et al.  From generic to specific deep representations for visual recognition , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Xinbo Gao,et al.  A deep feature based framework for breast masses classification , 2016, Neurocomputing.

[12]  Dean C. Barratt,et al.  Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks , 2017, CMMI/RAMBO/SWITCH@MICCAI.

[13]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Yunchao Wei,et al.  Perceptual Generative Adversarial Networks for Small Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Eric P. Xing,et al.  SCAN: Structure Correcting Adversarial Network for Organ Segmentation in Chest X-Rays , 2017, DLMIA/ML-CDS@MICCAI.

[16]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[17]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[18]  David Dagan Feng,et al.  Synthesis of Positron Emission Tomography (PET) Images via Multi-channel Generative Adversarial Networks (GANs) , 2017, CMMI/RAMBO/SWITCH@MICCAI.

[19]  Christoph Meinel,et al.  Conditional Adversarial Network for Semantic Segmentation of Brain Tumor , 2017, ArXiv.

[20]  Hassan Mathkour,et al.  Optimized Gabor features for mass classification in mammography , 2016, Appl. Soft Comput..

[21]  Dacheng Tao,et al.  Perceptual Adversarial Networks for Image-to-Image Transformation , 2017, IEEE Transactions on Image Processing.

[22]  Emily F Conant,et al.  Breast cancer screening using tomosynthesis in combination with digital mammography. , 2014, JAMA.

[23]  Fabián Narváez,et al.  Characterizing Architectural Distortion in Mammograms by Linear Saliency , 2017, Journal of Medical Systems.

[24]  Maryellen L. Giger,et al.  Breast image feature learning with adaptive deconvolutional networks , 2012, Medical Imaging.

[25]  Daguang Xu,et al.  Automatic Vertebra Labeling in Large-Scale 3D CT using Deep Image-to-Image Network with Message Passing and Sparsity Regularization , 2017, IPMI.

[26]  Song Han,et al.  Deep Generative Adversarial Networks for Compressed Sensing Automates MRI , 2017, ArXiv.

[27]  N. Dubrawsky.  Cancer statistics , 1989, CA: a cancer journal for clinicians.

[28]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[29]  A. Jemal,et al.  Breast cancer statistics, 2015: Convergence of incidence rates between black and white women , 2016, CA: a cancer journal for clinicians.

[30]  Gregory D. Hager,et al.  Adversarial deep structured nets for mass segmentation from mammograms , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[31]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[34]  Christopher Ré,et al.  Learning to Compose Domain-Specific Transformations for Data Augmentation , 2017, NIPS.

[35]  Sang Jun Park,et al.  Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks , 2017, ArXiv.

[36]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  C. K. Chua,et al.  Computer-Aided Breast Cancer Detection Using Mammograms: A Review , 2013, IEEE Reviews in Biomedical Engineering.

[38]  Huai Li,et al.  Artificial convolution neural network for medical image pattern recognition , 1995, Neural Networks.

[39]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[41]  A. Jemal,et al.  Cancer statistics, 2016 , 2016, CA: a cancer journal for clinicians.

[42]  Anjan Gudigar,et al.  Application of Gabor wavelet and Locality Sensitive Discriminant Analysis for automated identification of breast cancer using digitized mammogram images , 2016, Appl. Soft Comput..

[43]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[44]  Jonathan H Sunshine,et al.  How widely is computer-aided detection used in screening and diagnostic mammography? , 2010, Journal of the American College of Radiology : JACR.

[45]  Tao Xu,et al.  SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation , 2017, Neuroinformatics.

[46]  Muhammad Hussain,et al.  A comparison of different Gabor feature extraction approaches for mass classification in mammography , 2015, Multimedia Tools and Applications.

[47]  Rebecca L. Siegel Mph,et al.  Cancer statistics, 2016 , 2016 .

[48]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Sungroh Yoon,et al.  How Generative Adversarial Nets and its variants Work: An Overview of GAN , 2017, ArXiv.

[50]  Ping Tan,et al.  DualGAN: Unsupervised Dual Learning for Image-to-Image Translation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[51]  Nicolas Pinto,et al.  Why is Real-World Visual Object Recognition Hard? , 2008, PLoS Comput. Biol..

[52]  Kaiqi Huang,et al.  GP-GAN: Towards Realistic High-Resolution Image Blending , 2017, ACM Multimedia.

[53]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[54]  Xiaohui Xie,et al.  Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification , 2016, bioRxiv.

[55]  Shuihua Wang,et al.  Computer-aided diagnosis of abnormal breasts in mammogram images by weighted-type fractional Fourier transform , 2016 .

[56]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[57]  Daniel L. Rubin,et al.  Optimizing and Visualizing Deep Learning for Benign/Malignant Classification in Breast Tumors , 2017, ArXiv.

[58]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[59]  John T. Guibas,et al.  Synthetic Medical Images from Dual Generative Adversarial Networks , 2017, ArXiv.

[60]  R Nithya,et al.  Classification of Normal and Abnormal Patterns in Digital Mammograms for Diagnosis of Breast Cancer , 2011 .

[61]  Anselmo Cardoso de Paiva,et al.  Detection of masses in mammogram images using CNN, geostatistic functions and SVM , 2011, Comput. Biol. Medicine.

[62]  Richard H. Moore,et al.  THE DIGITAL DATABASE FOR SCREENING MAMMOGRAPHY , 2007 .