Deep Learning Using Havrda-Charvat Entropy for Classification of Pulmonary Optical Endomicroscopy

Abstract 1) Objective Pulmonary optical endomicroscopy (POE) is an imaging technology in real time. It allows to examine pulmonary alveoli at a microscopic level. Acquired in clinical settings, a POE image sequence can have as much as 25% of the sequence being uninformative frames (i.e. pure-noise and motion artifacts). For future data analysis, these uninformative frames must be first removed from the sequence. Therefore, the objective of our work is to develop an automatic detection method of uninformative images in endomicroscopy images. 2) Material and methods We propose to take the detection problem as a classification one. Considering advantages of deep learning methods, a classifier based on CNN (Convolutional Neural Network) is designed with a new loss function based on Havrda-Charvat entropy which is a parametrical generalization of the Shannon entropy. We propose to use this formula to get a better hold on all sorts of data since it provides a model more stable than the Shannon entropy. 3) Results Our method is tested on one POE dataset including 3895 distinct images and is showing better results than using Shannon entropy and behaves better with regard to the problem of overfitting. We obtain 70% of accuracy with Shannon entropy versus 77 to 79% with Havrda-Charvat. 4) Conclusion We can conclude that Havrda-Charvat entropy is better suited for restricted and or noisy datasets due to its generalized nature. It is also more suitable for classification in endomicroscopy datasets.

[1]  Tao Jiang,et al.  Minimum entropy clustering and applications to gene expression analysis , 2004, Proceedings. 2004 IEEE Computational Systems Bioinformatics Conference, 2004. CSB 2004..

[2]  Dinggang Shen,et al.  Deep ensemble learning of sparse regression models for brain disease diagnosis , 2017, Medical Image Anal..

[3]  Charles E McCulloch,et al.  Relaxing the rule of ten events per variable in logistic and Cox regression. , 2007, American journal of epidemiology.

[4]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[5]  Gurdas Ram,et al.  A Generalization of the Havrda-Charvat and Tsallis Entropy and Its Axiomatic Characterization , 2014 .

[6]  Hao Wu,et al.  Superpixels for Spatially Reinforced Bayesian Classification of Hyperspectral Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[7]  Pierre Lanchantin,et al.  Unsupervised segmentation of triplet Markov chains hidden with long-memory noise , 2008, Signal Process..

[8]  Nicolas Audebert,et al.  Multimodal deep networks for text and image-based document classification , 2019, PKDD/ECML Workshops.

[9]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[10]  David Dagan Feng,et al.  X-ray image classification using domain transferred convolutional neural networks and local sparse spatial pyramid , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[11]  Haruna Chiroma,et al.  Machine learning for email spam filtering: review, approaches and open research problems , 2019, Heliyon.

[12]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[13]  K. Thangavel,et al.  Mammogram Image Classification : Non-Shannon Entropy based Ant-Miner , 2015 .

[14]  Peter Sollich,et al.  Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities , 2002, Machine Learning.

[15]  S.-M. Jung,et al.  Hyers-Ulam stability of linear differential equations of first order , 2004, Appl. Math. Lett..

[17]  Gyula Maksa.  The stability of the entropy of degree alpha , 2008 .

[18]  Vasif V. Nabiyev,et al.  A novel automatic suspicious mass regions identification using Havrda & Charvat entropy and Otsu's N thresholding , 2014, Comput. Methods Programs Biomed..

[19]  P. Lambin,et al.  Machine Learning methods for Quantitative Radiomic Biomarkers , 2015, Scientific Reports.

[20]  M. Basseville.  Information : entropies, divergences et moyennes , 1996 .

[21]  S. N. Omkar,et al.  A Supervised Learning Methodology for Real-Time Disguised Face Recognition in the Wild , 2018, ArXiv.

[22]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[23]  Ting Chen,et al.  Group-Wise Point-Set Registration Using a Novel CDF-Based Havrda-Charvát Divergence , 2009, International Journal of Computer Vision.

[24]  Nikolaos Limnios,et al.  Maximum likelihood estimation for hidden semi-Markov models , 2006 .

[25]  Qiuyu Zhu,et al.  A New Loss Function for CNN Classifier Based on Predefined Evenly-Distributed Class Centroids , 2019, IEEE Access.

[26]  A. Amyar,et al.  3-D RPET-NET: Development of a 3-D PET Imaging Convolutional Neural Network for Radiomics Analysis and Outcome Prediction , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[27]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[28]  Stephen McLaughlin,et al.  Automated Detection of Uninformative Frames in Pulmonary Optical Endomicroscopy , 2017, IEEE Transactions on Biomedical Engineering.

[29]  Chan Basaruddin,et al.  A review on conditional random fields as a sequential classifier in machine learning , 2017, 2017 International Conference on Electrical Engineering and Computer Science (ICECOS).

[30]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[31]  Shivajirao M. Jadhav,et al.  Deep convolutional neural network based medical image classification for disease diagnosis , 2019, Journal of Big Data.

[32]  Vladimir Vapnik,et al.  Support-vector networks , 2004, Machine Learning.

[33]  P. Kiessler,et al.  An Introduction to Bayesian Analysis: Theory and Methods , 2008 .

[34]  Gautam Kunapuli,et al.  Conditional Random Fields For Brain Tissue Segmentation , 2013 .

[35]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[36]  Satish Kumar,et al.  A Coding Theorem on Havrda-Charvat and Tsallis's Entropy , 2012 .

[37]  Emanuele Frontoni,et al.  Supervised CNN Strategies for Optical Image Segmentation and Classification in Interventional Medicine , 2020 .

[38]  Su Ruan,et al.  A review: Deep learning for medical image segmentation using multi-modality fusion , 2019, Array.

[39]  R. E. Edwards,et al.  Functional Analysis: Theory and Applications , 1965 .

[40]  Gordon Wetzstein,et al.  Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification , 2018, Scientific Reports.

[41]  Mingjun Liu,et al.  Classification of Optical Remote Sensing Images Based on Convolutional Neural Network , 2019, 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT).

[42]  G. Bourg-Heckly,et al.  In vivo assessment of the pulmonary microcirculation in elastase-induced emphysema using probe-based confocal fluorescence microscopy , 2012 .