Combining DC-GAN with ResNet for blood cell image classification

In medicine, white blood cells (WBCs) play an important role in the human immune system. The different types of WBC abnormalities are related to different diseases so that the total number and classification of WBCs are critical for clinical diagnosis and therapy. However, the traditional method of white blood cell classification is to segment the cells, extract features, and then classify them. Such method depends on the good segmentation, and the accuracy is not high. Moreover, the insufficient data or unbalanced samples can cause the low classification accuracy of model by using deep learning in medical diagnosis. To solve these problems, this paper proposes a new blood cell image classification framework which is based on a deep convolutional generative adversarial network (DC-GAN) and a residual neural network (ResNet). In particular, we introduce a new loss function which is improved the discriminative power of the deeply learned features. The experiments show that our model has a good performance on the classification of WBC images, and the accuracy reaches 91.7%. Graphical Abstract Overview of the proposed method, we use the deep convolution generative adversarial networks (DC-GAN) to generate new samples that are used as supplementary input to a ResNet, the transfer learning method is used to initialize the parameters of the network, the output of the DC-GAN and the parameters are applied the final classification network. In particular, we introduced a modified loss function for classification to increase inter-class variations and decrease intra-class differences.

[1]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[2]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[5]  Surjya K. Pal,et al.  Statistical pattern analysis of white blood cell nuclei morphometry , 2010, 2010 IEEE Students Technology Symposium (TechSym).

[6]  Takuya Akiba,et al.  Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes , 2017, ArXiv.

[7]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[8]  Lixin Zheng,et al.  Combining Convolutional Neural Network With Recursive Neural Network for Blood Cell Image Classification , 2018, IEEE Access.

[9]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[10]  Mu-Chun Su,et al.  A Neural-Network-Based Approach to White Blood Cell Classification , 2014, TheScientificWorldJournal.

[11]  Yong Xia,et al.  Attention Residual Learning for Skin Lesion Classification , 2019, IEEE Transactions on Medical Imaging.

[12]  Jaroonrut Prinyakupt,et al.  Segmentation of white blood cells and comparison of cell morphology by linear and naïve Bayes classifiers , 2015, BioMedical Engineering OnLine.

[13]  René Vidal,et al.  Multi-cell Detection and Classification Using a Generative Convolutional Model , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Zahoor Jan,et al.  Color Based Segmentation of White Blood Cells in Blood Photomicrographs Using Image Quantization , 2014 .

[16]  Hanqi Zhuang,et al.  Classification of White Blood Cells with PatternNet-fused Ensemble of Convolutional Neural Networks (PECNN) , 2018, 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[17]  Li Sun,et al.  Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images , 2019 .

[18]  Rong-Ming Chen,et al.  Blood Cell Image Classification Based on Hierarchical SVM , 2011, 2011 IEEE International Symposium on Multimedia.

[19]  Cheng Lei,et al.  Exploring the Deep Feature Space of a Cell Classification Neural Network , 2018, ArXiv.

[20]  Vandana Jagtap,et al.  Computer Aided Melanoma Skin Cancer Detection Using Image Processing , 2015 .

[21]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[22]  Gang Pan,et al.  CSNN: An Augmented Spiking based Framework with Perceptron-Inception , 2018, IJCAI.

[23]  Yao Li,et al.  Optogenetic Excitation of Ipsilesional Sensorimotor Neurons is Protective in Acute Ischemic Stroke: A Laser Speckle Imaging Study , 2019, IEEE Transactions on Biomedical Engineering.

[24]  Jasmine Banks,et al.  Classification of white blood cell types from microscope images: Techniques and challenges , 2018 .

[25]  Shanbao Tong,et al.  Monitoring Acute Stroke in Mouse Model Using Laser Speckle Imaging-Guided Visible-Light Optical Coherence Tomography , 2018, IEEE Transactions on Biomedical Engineering.

[26]  Gang Pan,et al.  Deep CovDenseSNN: A hierarchical event-driven dynamic framework with spiking neurons in noisy environment , 2019, Neural Networks.

[27]  Subhransu Maji,et al.  3D Shape Induction from 2D Views of Multiple Objects , 2016, 2017 International Conference on 3D Vision (3DV).

[28]  Hamid Soltanian-Zadeh,et al.  Automatic Recognition of Five Types of White Blood Cells in Peripheral Blood , 2010, ICIAR.

[29]  Ming Zhang,et al.  Overfitting remedy by sparsifying regularization on fully-connected layers of CNNs , 2019, Neurocomputing.

[30]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[31]  Anton Schwaighofer,et al.  Learning Gaussian Process Kernels via Hierarchical Bayes , 2004, NIPS.

[32]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[33]  Alireza Karimian,et al.  Design a new algorithm to count white blood cells for classification leukemic blood image using machine vision system , 2016, 2016 6th International Conference on Computer and Knowledge Engineering (ICCKE).

[34]  Jianwei Zhao,et al.  Automatic detection and classification of leukocytes using convolutional neural networks , 2016, Medical & Biological Engineering & Computing.

[35]  Sini Shibu,et al.  Analysis of blood samples for counting leukemia cells using Support vector machine and nearest neighbour , 2014 .

[36]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[37]  Yann LeCun,et al.  Deep multi-scale video prediction beyond mean square error , 2015, ICLR.

[38]  A. G. Ramakrishnan,et al.  Automation of differential blood count , 2003, TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region.

[39]  Klaus H. Maier-Hein,et al.  No New-Net , 2018, 1809.10483.

[40]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[41]  Ashish Khanna,et al.  Optimized Binary Bat algorithm for classification of white blood cells , 2019, Measurement.

[42]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[43]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Edwin V. Bonilla,et al.  Multi-task Gaussian Process Prediction , 2007, NIPS.

[45]  Wen-Huang Cheng,et al.  Computer-aided classification of lung nodules on computed tomography images via deep learning technique , 2015, OncoTargets and therapy.

[46]  Qi Wu,et al.  Medical image classification using synergic deep learning , 2019, Medical Image Anal..

[47]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[48]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[49]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.

[50]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Kibok Lee,et al.  Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples , 2017, ICLR.

[52]  Daniel Rueckert,et al.  Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II , 2017, Lecture Notes in Computer Science.

[53]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[54]  M. Y. Mashor,et al.  Classification of blasts in acute leukemia blood samples using k-nearest neighbour , 2012, 2012 IEEE 8th International Colloquium on Signal Processing and its Applications.

[55]  Yifan Yu,et al.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison , 2019, AAAI.

[56]  Gang Pan,et al.  Character recognition from trajectory by recurrent spiking neural networks , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[57]  K. V. Arya,et al.  Feature selection and classification of leukocytes using random forest , 2014, Medical & Biological Engineering & Computing.

[58]  Jie Xu,et al.  The practical implementation of artificial intelligence technologies in medicine , 2019, Nature Medicine.

[59]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.