Automatic diagnosis of fungal keratitis using data augmentation and image fusion with deep convolutional neural network

BACKGROUND AND OBJECTIVES Fungal keratitis is caused by inflammation of the cornea that results from infection by fungal organisms. The lack of an early effective diagnosis often results in serious complications even blindness. Confocal microscopy is one of the most effective methods in the diagnosis of fungal keratitis, but the diagnosis depends on the subjective judgment of medical experts. METHODS To address this problem, this paper proposes a novel convolutional neural network framework for the automatic diagnosis of fungal keratitis using data augmentation and image fusion. Firstly, a normal image is augmented by flipping to solve the problem of having a limited and imbalanced database. Secondly, a sub-area contrast stretching algorithm is proposed for image preprocessing to highlight the key structures in the images and to filter out irrelevant information. Thirdly, the histogram matching fusion algorithm is implemented, then the preprocessed image is fused with the original image to form a new algorithm framework and a new database. Finally, the traditional convolutional neural network is integrated into the novel algorithm framework to perform the experiments. RESULTS Experiments show that the accuracy of traditional AlexNet and VGGNet is 99.35% and 99.14%, that of AlexNet and VGGNet based on MF fusion is 99.80% and 99.83%, and that of AlexNet and VGGNet based on histogram matching fusion (HMF) is 99.95% and 99.89%. The experimental results show that the AlexNet framework using data augmentation and image fusion achieves a perfect trade-off between the diagnostic performance and the computational complexity, with a diagnostic accuracy of 99.95%. CONCLUSIONS These experimental results demonstrate the novel convolutional neural network framework perfectly balances the diagnostic performance and computational complexity, and can improve the effect and real-time performance in the diagnosis of fungal keratitis.

[1]  Xiaoxia Yin,et al.  Accurate Image Analysis of the Retina Using Hessian Matrix and Binarisation of Thresholded Entropy with Application of Texture Mapping , 2014, PloS one.

[2]  Fatemeh Keshavarz,et al.  Comparative Study of Gram Stain, Potassium Hydroxide Smear, Culture and Nested PCR in the Diagnosis of Fungal Keratitis , 2010, Ophthalmic Research.

[3]  David S Chu,et al.  Surgical Outcomes in Cases of Contact Lens-Related Fusarium Keratitis , 2010, Eye & contact lens.

[4]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  P. Thomas,et al.  Fungal infections of the cornea , 2003, Eye.

[6]  J. Younger,et al.  Statistical methodology: III. Receiver operating characteristic (ROC) curves. , 1997, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[7]  D K PISCHEL,et al.  Detachment of the vitreous as seen with slit-lamp examination. , 1952, Transactions of the American Ophthalmological Society.

[8]  C J Koester,et al.  Scanning slit confocal microscopy of fungal keratitis. , 1997, Archives of ophthalmology.

[9]  Dongmei Wei,et al.  Automatic Detecting Cornea Fungi Based on Texture Analysis , 2016, 2016 IEEE International Conference on Smart Cloud (SmartCloud).

[10]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[11]  J. Pawley,et al.  Handbook of Biological Confocal Microscopy , 1990, Springer US.

[12]  Wilpen L. Gorr,et al.  Forecast accuracy measures for exception reporting using receiver operating characteristic curves , 2009 .

[13]  Ching-Chung Yang.  Image enhancement by modified contrast-stretching manipulation , 2006 .

[14]  Du Rui.  Establishment of rabbit fungal (aspergillus) keratitis model and corneal examination with confocal microscope , 2006 .

[15]  Abhisek Ukil,et al.  Support Vector Machine , 2007 .

[16]  Xiabi Liu,et al.  Hybrid resampling and multi-feature fusion for automatic recognition of cavity imaging sign in lung CT , 2019, Future Gener. Comput. Syst..

[17]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[18]  Jianwen Luo,et al.  Robust Segmentation of Intima–Media Borders With Different Morphologies and Dynamics During the Cardiac Cycle , 2018, IEEE Journal of Biomedical and Health Informatics.

[19]  Xinyi Wu,et al.  Hyphae Detection in Fungal Keratitis Images With Adaptive Robust Binary Pattern , 2018, IEEE Access.

[20]  K Meng.  Multi-disciplinary Intersection and Integration:the Development Tendency and the Change in Meteorology in Colleges , 2012 .

[21]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[22]  D. T. Mane,et al.  A Survey on Supervised Convolutional Neural Network and Its Major Applications , 2017, Int. J. Rough Sets Data Anal..

[23]  Nello Cristianini,et al.  Support vector machine classification and validation of cancer tissue samples using microarray expression data , 2000, Bioinform..

[24]  Bertrand Zavidovique,et al.  Rotationally Invariant Hashing of Median Binary Patterns for Texture Classification , 2008, ICIAR.

[25]  Amit Singh,et al.  Confocal microscopy: a powerful technique for biological research , 1998 .

[26]  Huafeng Liu,et al.  Direct delineation of myocardial infarction without contrast agents using a joint motion feature learning architecture , 2018, Medical Image Anal..

[27]  Dong Liang,et al.  Motion Tracking of the Carotid Artery Wall From Ultrasound Image Sequences: a Nonlinear State-Space Approach , 2018, IEEE Transactions on Medical Imaging.

[28]  Weiyun Shi,et al.  Spectrum of fungal keratitis in north China. , 2006, Ophthalmology.

[29]  Heye Zhang,et al.  Direct and simultaneous estimation of cardiac four chamber volumes by multioutput sparse regression , 2017, Medical Image Anal..

[30]  S. Srivannaboon,et al.  Normal Human Corneal Cell Populations Evaluated by In Vivo Scanning Slit Confocal Microscopy , 1998, Cornea.

[31]  Xiaogang Wang,et al.  Medical image classification with convolutional neural network , 2014, 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV).

[32]  Sahibsingh A. Dudani.  The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.