A 3D attention networks for classification of white blood cells from microscopy hyperspectral images

Abstract Peripheral blood smear analysis is a key process for hematologists to reflect condition of human immune system and one of the earliest clinical applications that benefit from automatic computer-aided analysis. Changes in the ratio of white blood cell (WBC) types are associated with the determination of blood diseases, and therefore accurate classification assures accurate therapy. In this study, we applied deep convolution networks on microscopy hyperspectral images for WBC classification. We proposed a 3D convolutional networks named deep hyper that enables learning spectral and spatial features by itself to make fully use of three dimensional hyperspectral data for white blood cell classification. In order to enhance the representative power of the model in an efficient manner, we integrated a 3D attention module with the last block of the model to put more emphasis on important features. The overall classification accuracy of deep hyper can achieve 96% compared to 90% of the machine learning based method and the attention module allows deep hyper to achieve the best performance of 97.72%. Furthermore, we explored the correlation between spectral features and WBC classification performances to present that hyperspectral characteristics play an important role in classifying specific type of WBC. These findings demonstrate that combing microscopy hyperspectral image with deep convolution networks is beneficial for blood smear analysis especially white blood cell classification.

[1]  Qian Huang,et al.  Blood Cell Classification Based on Hyperspectral Imaging With Modulated Gabor and CNN , 2020, IEEE Journal of Biomedical and Health Informatics.

[2]  Qingli Li,et al.  Quantitative analysis of liver tumors at different stages using microscopic hyperspectral imaging technology , 2018, Journal of biomedical optics.

[3]  Amy Y. Chen,et al.  Detection and delineation of squamous neoplasia with hyperspectral imaging in a mouse model of tongue carcinogenesis , 2018, Journal of biophotonics.

[4]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Guolan Lu,et al.  Medical hyperspectral imaging: a review , 2014, Journal of biomedical optics.

[6]  P. S. Hiremath,et al.  Automatic classification of bacterial cells in digital microscopic images , 2010, International Conference on Digital Image Processing.

[7]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[9]  Shuying Shen,et al.  Fine-grained leukocyte classification with deep residual learning for microscopic images , 2018, Comput. Methods Programs Biomed..

[10]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[11]  Mengmeng Zhang,et al.  Medical Hyperspectral Image Classification Based on End-to-End Fusion Deep Neural Network , 2019, IEEE Transactions on Instrumentation and Measurement.

[12]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[13]  David A. Landgrebe,et al.  Hyperspectral image data analysis , 2002, IEEE Signal Process. Mag..

[14]  P. Mirabelli,et al.  Raman detection and identification of normal and leukemic hematopoietic cells , 2018, Journal of biophotonics.

[15]  Keerthana Prasad,et al.  Comparison of traditional image processing and deep learning approaches for classification of white blood cells in peripheral blood smear images , 2019, Biocybernetics and Biomedical Engineering.

[16]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[17]  Mei Zhou,et al.  A spectral and morphologic method for white blood cell classification , 2016 .

[18]  Jon Atli Benediktsson,et al.  Advances in Spectral-Spatial Classification of Hyperspectral Images , 2013, Proceedings of the IEEE.

[19]  Shivajirao M. Jadhav,et al.  Deep convolutional neural network based medical image classification for disease diagnosis , 2019, Journal of Big Data.

[20]  Osman Kalender,et al.  Automatic segmentation, counting, size determination and classification of white blood cells , 2014 .

[21]  Mira Barak,et al.  The rate of manual peripheral blood smear reviews in outpatients , 2009, Clinical chemistry and laboratory medicine.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Lin Yang,et al.  Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review , 2016, IEEE Reviews in Biomedical Engineering.

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Guolan Lu,et al.  Deep convolutional neural networks for classifying head and neck cancer using hyperspectral imaging , 2017, Journal of biomedical optics.

[26]  Jin Woo Choi,et al.  White blood cell differential count of maturation stages in bone marrow smear using dual-stage convolutional neural networks , 2017, PloS one.

[27]  K. V. Arya,et al.  Automated microscopic image analysis for leukocytes identification: a survey. , 2014, Micron.

[28]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[29]  Zvi Malik,et al.  Chromatin Condensation in Erythropoiesis Resolved by Multipixel Spectral Imaging: Differentiation Versus Apoptosis , 1997, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[30]  Mei Zhou,et al.  Skin cells segmentation algorithm based on spectral angle and distance score , 2015 .

[31]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[32]  G. Zito,et al.  [INVITED] Raman microscopy based sensing of leukemia cells: A review , 2018, Optics & Laser Technology.

[33]  Lipo Wang,et al.  Image Thresholding Improves 3-Dimensional Convolutional Neural Network Diagnosis of Different Acute Brain Hemorrhages on Computed Tomography Scans , 2019, Sensors.

[34]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[35]  Dan Savastru,et al.  Hyperspectral Imaging in the Medical Field: Present and Future , 2014 .

[36]  R. Ornberg,et al.  Analysis of Stained Objects in Histological Sections by Spectral Imaging and Differential Absorption , 1999, The journal of histochemistry and cytochemistry : official journal of the Histochemistry Society.

[37]  Mei Zhou,et al.  Red Blood Cell Count Automation Using Microscopic Hyperspectral Imaging Technology , 2015, Applied spectroscopy.

[38]  Yeqi Bai,et al.  Automated brain histology classification using machine learning , 2019, Journal of Clinical Neuroscience.

[39]  Yanhui Guo,et al.  White blood cells identification system based on convolutional deep neural learning networks , 2017, Comput. Methods Programs Biomed..

[40]  Gustavo Marrero Callicó,et al.  Deep Learning-Based Framework for In Vivo Identification of Glioblastoma Tumor using Hyperspectral Images of Human Brain , 2019, Sensors.

[41]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Dongsheng Wang,et al.  A Minimum Spanning Forest-Based Method for Noninvasive Cancer Detection With Hyperspectral Imaging , 2016, IEEE Transactions on Biomedical Engineering.

[43]  Jaroonrut Prinyakupt,et al.  Segmentation of white blood cells and comparison of cell morphology by linear and naïve Bayes classifiers , 2015, BioMedical Engineering OnLine.