Deep learning using Havrda-Charvat entropy for classification of pulmonary endomicroscopy

Pulmonary optical endomicroscopy (POE) is an imaging technology in real time. It allows to examine pulmonary alveoli at a microscopic level. Acquired in clinical settings, a POE image sequence can have as much as 25% of the sequence being uninformative frames (i.e. pure-noise and motion artefacts). For future data analysis, these uninformative frames must be first removed from the sequence. Therefore, the objective of our work is to develop an automatic detection method of uninformative images in endomicroscopy images. We propose to take the detection problem as a classification one. Considering advantages of deep learning methods, a classifier based on CNN (Convolutional Neural Network) is designed with a new loss function based on Havrda-Charvat entropy which is a parametrical generalization of the Shannon entropy. We propose to use this formula to get a better hold on all sorts of data since it provides a model more stable than the Shannon entropy. Our method is tested on one POE dataset including 2947 distinct images, is showing better results than using Shannon entropy and behaves better with regard to the problem of overfitting.

[1]  Shivajirao M. Jadhav,et al.  Deep convolutional neural network based medical image classification for disease diagnosis , 2019, Journal of Big Data.

[2]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[3]  David Dagan Feng,et al.  X-ray image classification using domain transferred convolutional neural networks and local sparse spatial pyramid , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[4]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[5]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[6]  Qiuyu Zhu,et al.  A New Loss Function for CNN Classifier Based on Predefined Evenly-Distributed Class Centroids , 2019, IEEE Access.

[7]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[8]  Hao Wu,et al.  Superpixels for Spatially Reinforced Bayesian Classification of Hyperspectral Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[9]  Pierre Lanchantin,et al.  Unsupervised segmentation of triplet Markov chains hidden with long-memory noise , 2008, Signal Process..

[10]  Nicolas Audebert,et al.  Multimodal deep networks for text and image-based document classification , 2019, PKDD/ECML Workshops.

[11]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[12]  K. Thangavel,et al.  Mammogram Image Classification : Non-Shannon Entropy based Ant-Miner , 2015 .

[13]  Peter Sollich,et al.  Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities , 2002, Machine Learning.

[14]  G. Bourg-Heckly,et al.  In vivo assessment of the pulmonary microcirculation in elastase-induced emphysema using probe-based confocal fluorescence microscopy , 2012 .

[15]  M. Basseville.  Information : entropies, divergences et moyennes , 1996 .

[16]  Su Ruan,et al.  A review: Deep learning for medical image segmentation using multi-modality fusion , 2019, Array.

[17]  A. Amyar,et al.  3-D RPET-NET: Development of a 3-D PET Imaging Convolutional Neural Network for Radiomics Analysis and Outcome Prediction , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[18]  Tao Jiang,et al.  Minimum entropy clustering and applications to gene expression analysis , 2004, Proceedings. 2004 IEEE Computational Systems Bioinformatics Conference, 2004. CSB 2004..

[19]  Jayanta K. Ghosh,et al.  An Introduction to Bayesian Analysis , 2006 .

[20]  Dinggang Shen,et al.  Deep ensemble learning of sparse regression models for brain disease diagnosis , 2017, Medical Image Anal..

[21]  Haruna Chiroma,et al.  Machine learning for email spam filtering: review, approaches and open research problems , 2019, Heliyon.

[22]  Gyula Maksa.  The stability of the entropy of degree alpha , 2008 .

[23]  Vasif V. Nabiyev,et al.  A novel automatic suspicious mass regions identification using Havrda & Charvat entropy and Otsu's N thresholding , 2014, Comput. Methods Programs Biomed..

[24]  S.-M. Jung,et al.  Hyers-Ulam stability of linear differential equations of first order , 2004, Appl. Math. Lett..

[25]  Stephen McLaughlin,et al.  Automated Detection of Uninformative Frames in Pulmonary Optical Endomicroscopy , 2017, IEEE Transactions on Biomedical Engineering.

[26]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[27]  Chan Basaruddin,et al.  A review on conditional random fields as a sequential classifier in machine learning , 2017, 2017 International Conference on Electrical Engineering and Computer Science (ICECOS).

[28]  Lakhmi C. Jain,et al.  An Introduction to Deep Learners and Deep Learner Descriptors for Medical Applications , 2020 .

[29]  S. N. Omkar,et al.  A Supervised Learning Methodology for Real-Time Disguised Face Recognition in the Wild , 2018, ArXiv.

[30]  Gordon Wetzstein,et al.  Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification , 2018, Scientific Reports.

[31]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[32]  Mingjun Liu,et al.  Classification of Optical Remote Sensing Images Based on Convolutional Neural Network , 2019, 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT).

[33]  G. Fitzgerald,et al.  'I. , 2019, Australian journal of primary health.

[34]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[36]  Ting Chen,et al.  Group-Wise Point-Set Registration Using a Novel CDF-Based Havrda-Charvát Divergence , 2009, International Journal of Computer Vision.

[37]  Nikolaos Limnios,et al.  Maximum likelihood estimation for hidden semi-Markov models , 2006 .

[38]  Gautam Kunapuli,et al.  Conditional Random Fields For Brain Tissue Segmentation , 2013 .

[39]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[40]  Satish Kumar,et al.  A Coding Theorem on Havrda-Charvat and Tsallis's Entropy , 2012 .

[41]  Emanuele Frontoni,et al.  Supervised CNN Strategies for Optical Image Segmentation and Classification in Interventional Medicine , 2020 .

[42]  Charles E McCulloch,et al.  Relaxing the rule of ten events per variable in logistic and Cox regression. , 2007, American journal of epidemiology.

[43]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[44]  Gurdas Ram,et al.  A Generalization of the Havrda-Charvat and Tsallis Entropy and Its Axiomatic Characterization , 2014 .