Circumpapillary OCT-Focused Hybrid Learning for Glaucoma Grading Using Tailored Prototypical Neural Networks

Glaucoma is one of the leading causes of blindness worldwide and Optical Coherence Tomography (OCT) is the quintessential imaging technique for its detection. Unlike most of the state-of-the-art studies focused on glaucoma detection, in this paper, we propose, for the first time, a novel framework for glaucoma grading using raw circumpapillary B-scans. In particular, we set out a new OCT-based hybrid network which combines hand-driven and deep learning algorithms. An OCT-specific descriptor is proposed to extract hand-crafted features related to the retinal nerve fibre layer (RNFL). In parallel, an innovative CNN is developed using skip-connections to include tailored residual and attention modules to refine the automatic features of the latent space. The proposed architecture is used as a backbone to conduct a novel few-shot learning based on static and dynamic prototypical networks. The k-shot paradigm is redefined giving rise to a supervised end-to-end system which provides substantial improvements discriminating between healthy, early and advanced glaucoma samples. The training and evaluation processes of the dynamic prototypical network are addressed from two fused databases acquired via Heidelberg Spectralis system. Validation and testing results reach a categorical accuracy of 0.9459 and 0.8788 for glaucoma grading, respectively. Besides, the high performance reported by the proposed model for glaucoma detection deserves a special mention. The findings from the class activation maps are directly in line with the clinicians' opinion since the heatmaps pointed out the RNFL as the most relevant structure for glaucoma diagnosis.

[1]  J. Hornegger,et al.  Retinal Nerve Fiber Layer Segmentation on FD-OCT Scans of Normal Subjects and Glaucoma Patients , 2010, Biomedical optics express.

[2]  Stanislav Fort,et al.  Gaussian Prototypical Networks for Few-Shot Learning on Omniglot , 2017, ArXiv.

[3]  Junhua Wang,et al.  Prototypical Siamese Networks for Few-shot Learning , 2020, 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication (ICEIEC).

[4]  T. Wong,et al.  Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis. , 2014, Ophthalmology.

[5]  Shafin Rahman,et al.  An Approach for Automated Segmentation of Retinal Layers In Peripapillary Spectralis SDOCT Images Using Curve Regularisation , 2017 .

[6]  W. Feuer,et al.  Frequency of abnormal retinal nerve fibre layer and ganglion cell layer SDOCT scans in healthy eyes and glaucoma suspects in a prospective longitudinal study , 2013, British Journal of Ophthalmology.

[7]  Hao Chen,et al.  Unifying Structure Analysis and Surrogate-driven Function Regression for Glaucoma OCT Image Screening , 2019, MICCAI.

[8]  Rohit Varma,et al.  Deep Learning Classifiers for Automated Detection of Gonioscopic Angle Closure Based on Anterior Segment OCT Images. , 2019, American journal of ophthalmology.

[9]  F. Horn,et al.  Longitudinal analysis of progression in glaucoma using spectral-domain optical coherence tomography. , 2013, Investigative ophthalmology & visual science.

[10]  Zhiyuan Liu,et al.  Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification , 2019, AAAI.

[11]  Felipe A. Medeiros,et al.  Quantification of Retinal Nerve Fibre Layer Thickness on Optical Coherence Tomography with a Deep Learning Segmentation-Free Approach , 2020, Scientific Reports.

[12]  R. Weinreb,et al.  Impact of age-related change of retinal nerve fiber layer and macular thicknesses on evaluation of glaucoma progression. , 2013, Ophthalmology.

[13]  Gadi Wollstein,et al.  OCT for glaucoma diagnosis, screening and detection of glaucoma progression , 2013, British Journal of Ophthalmology.

[14]  Oscar Camacho Nieto,et al.  A machine learning approach to medical image classification: Detecting age-related macular degeneration in fundus images , 2017, Comput. Electr. Eng..

[15]  Mohammad I. Daoud,et al.  Content-based Image Retrieval for Breast Ultrasound Images using Convolutional Autoencoders: A Feasibility Study , 2019, 2019 3rd International Conference on Bio-engineering for Smart Technologies (BioSMART).

[16]  Changshui Zhang,et al.  Boosting Few-Shot Image Recognition Via Domain Alignment Prototypical Networks , 2018, 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI).

[17]  Shengli Sun,et al.  Hierarchical Attention Prototypical Networks for Few-Shot Text Classification , 2019, EMNLP.

[18]  Valery Naranjo,et al.  Analysis of Hand-Crafted and Automatic-Learned Features for Glaucoma Detection Through Raw Circumpapillary OCT Images , 2020, IDEAL.

[19]  Stephen Lin,et al.  A Deep Learning System for Automated Angle-Closure Detection in Anterior Segment Optical Coherence Tomography Images. , 2019, American journal of ophthalmology.

[20]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Hiroshi Ishikawa,et al.  A feature agnostic approach for glaucoma detection in OCT volumes , 2018, PloS one.

[22]  Mathias Lux,et al.  Dimensionality Reduction for Image Features using Deep Learning and Autoencoders , 2017, CBMI.

[23]  Xinjian Chen,et al.  Comparison of Retinal Thickness Measurements between the Topcon Algorithm and a Graph-Based Algorithm in Normal and Glaucoma Eyes , 2015, PloS one.

[24]  P. Khaw,et al.  Primary open-angle glaucoma , 2004, The Lancet.

[25]  Raphael Sznitman,et al.  A deep learning approach to automatic detection of early glaucoma from visual fields , 2018, PloS one.

[26]  Valery Naranjo,et al.  Glaucoma Detection From Raw Circumpapillary OCT Images Using Fully Convolutional Neural Networks , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[27]  Arslan Shaukat,et al.  Improved automated detection of glaucoma by correlating fundus and SD‐OCT image analysis , 2020, Int. J. Imaging Syst. Technol..

[28]  Paul Sajda,et al.  Enhancing the Accuracy of Glaucoma Detection from OCT Probability Maps using Convolutional Neural Networks , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[29]  Piotr Koniusz,et al.  Projective Subspace Networks For Few-Shot Learning , 2018 .

[30]  F. Medeiros,et al.  Comparing the Rates of Retinal Nerve Fiber Layer and Ganglion Cell-Inner Plexiform Layer Loss in Healthy Eyes and in Glaucoma Eyes. , 2017, American journal of ophthalmology.

[31]  Jae Y. Shin,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE transactions on medical imaging.

[32]  Yali Jia,et al.  Automated segmentation of peripapillary retinal boundaries in OCT combining a convolutional neural network and a multi-weights graph search. , 2019, Biomedical optics express.

[33]  Mehrtash Harandi,et al.  Adaptive Subspaces for Few-Shot Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  I. Scott,et al.  Expert agreement in evaluating the optic disc for glaucoma. , 1992, Ophthalmology.

[35]  Jian Zhang,et al.  Performance assessment of the deep learning technologies in grading glaucoma severity , 2018, ArXiv.

[36]  F. Medeiros,et al.  Detection of glaucoma progression with stratus OCT retinal nerve fiber layer, optic nerve head, and macular thickness measurements. , 2009, Investigative ophthalmology & visual science.

[37]  Carol Y. Cheung,et al.  Detecting Glaucoma Using 3D Convolutional Neural Network of Raw SD-OCT Optic Nerve Scans , 2019, ArXiv.

[38]  F. Medeiros,et al.  Detecting Structural Progression in Glaucoma with Optical Coherence Tomography. , 2017, Ophthalmology.

[39]  D. Hood,et al.  On improving the use of OCT imaging for detecting glaucomatous damage , 2014, British Journal of Ophthalmology.

[40]  J. Fujimoto,et al.  Optical Coherence Tomography , 1991 .

[41]  Neil J. Joshi,et al.  Automated Grading of Age-Related Macular Degeneration From Color Fundus Images Using Deep Convolutional Neural Networks , 2017, JAMA ophthalmology.

[42]  Hideo Yokota,et al.  Glaucoma Diagnosis with Machine Learning Based on Optical Coherence Tomography and Color Fundus Images , 2019, Journal of healthcare engineering.

[43]  Jingkuan Song,et al.  Binary Generative Adversarial Networks for Image Retrieval , 2017, AAAI.

[44]  M. Usman Akram,et al.  Data on OCT and fundus images for the detection of glaucoma , 2020, Data in brief.

[45]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[46]  Chong Wang,et al.  Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. , 2017, Biomedical optics express.

[47]  J. Caprioli,et al.  Optical coherence tomography to detect and manage retinal disease and glaucoma. , 2004, American journal of ophthalmology.

[48]  A. Peters,et al.  A Deep Learning Algorithm for Prediction of Age-Related Eye Disease Study Severity Scale for Age-Related Macular Degeneration from Color Fundus Photography. , 2018, Ophthalmology.

[49]  Jong Jin Jung,et al.  Rates and Patterns of Macular and Circumpapillary Retinal Nerve Fiber Layer Thinning in Preperimetric and Perimetric Glaucomatous Eyes , 2015, Journal of glaucoma.

[50]  Sejong Oh,et al.  Development of machine learning models for diagnosis of glaucoma , 2017, PloS one.

[51]  Chong-Wah Ngo,et al.  Transferrable Prototypical Networks for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[52]  Hao Chen,et al.  Towards multi-center glaucoma OCT image screening with semi-supervised joint structure and function multi-task learning , 2020, Medical Image Anal..

[53]  Robert N Weinreb,et al.  The structure and function relationship in glaucoma: implications for detection of progression and measurement of rates of change. , 2012, Investigative ophthalmology & visual science.

[54]  Robert N Weinreb,et al.  Retinal nerve fiber layer imaging with spectral-domain optical coherence tomography: patterns of retinal nerve fiber layer progression. , 2012, Ophthalmology.

[55]  Delia Cabrera DeBuc,et al.  Deep Learning based Retinal OCT Segmentation , 2018, Comput. Biol. Medicine.

[56]  Fei Li,et al.  Automatic differentiation of Glaucoma visual field from non-glaucoma visual filed using deep convolutional neural network , 2018, BMC Medical Imaging.

[57]  Alan F. Smeaton,et al.  Contrastive Representation Learning: A Framework and Review , 2020, IEEE Access.

[58]  Taimur Hassan,et al.  Clinically Verified Hybrid Deep Learning System for Retinal Ganglion Cells Aware Grading of Glaucomatous Progression , 2020, IEEE transactions on bio-medical engineering.

[59]  Giovanni Montesano,et al.  ReLayer: a Free, Online Tool for Extracting Retinal Thickness From Cross-Platform OCT Images , 2019, Translational vision science & technology.

[60]  Yuanjie Zheng,et al.  A Generative Model for OCT Retinal Layer Segmentation by Groupwise Curve Alignment , 2018, IEEE Access.

[61]  Masanori Hangai,et al.  Measurement of Retinal Nerve Fiber Layer Thickness and Macular Volume for Glaucoma Detection Using Optical Coherence Tomography , 2007, Japanese Journal of Ophthalmology.

[62]  Sangsoo Kim,et al.  A deep learning model for the detection of both advanced and early glaucoma using fundus photography , 2018, PloS one.

[63]  Remo Susanna Jr,et al.  Staging glaucoma patient: why and how? , 2009 .

[64]  Michael J. A. Girard,et al.  DRUNET: A Dilated-Residual U-Net Deep Learning Network to Digitally Stain Optic Nerve Head Tissues in Optical Coherence Tomography Images , 2018, Biomedical optics express.

[65]  Anders Heijl,et al.  Machine learning classifiers for glaucoma diagnosis based on classification of retinal nerve fibre layer thickness parameters measured by Stratus OCT , 2010, Acta ophthalmologica.

[66]  Ahmed E.Abd El-Naby,et al.  Correlation of retinal nerve fiber layer thickness and perimetric changes in primary open-angle glaucoma , 2018 .

[67]  Mona Kamal Abdellatif,et al.  Outer Retinal Layers' Thickness Changes in relation to Age and Choroidal Thickness in Normal Eyes , 2019, Journal of ophthalmology.

[68]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[69]  Weiwei Zhang,et al.  Automated retinal layers segmentation in SD-OCT images using dual-gradient and spatial correlation smoothness constraint , 2014, Comput. Biol. Medicine.

[70]  P. Lichter Variability of expert observers in evaluating the optic disc. , 1976, Transactions of the American Ophthalmological Society.

[71]  Vasudevan Lakshminarayanan,et al.  OCTID: Optical Coherence Tomography Image Database , 2018, Comput. Electr. Eng..

[72]  Adrián Colomer,et al.  Glaucoma Detection from Raw SD-OCT Volumes: A Novel Approach Focused on Spatial Dependencies , 2020, Comput. Methods Programs Biomed..

[73]  Felipe A Medeiros,et al.  Assessment of a Segmentation-Free Deep Learning Algorithm for Diagnosing Glaucoma From Optical Coherence Tomography Scans. , 2020, JAMA ophthalmology.

[74]  G. Holló,et al.  Detection of Early Glaucomatous Progression With Different Parameters of the RTVue Optical Coherence Tomograph , 2014, Journal of glaucoma.

[75]  Hye Jin Lee,et al.  Detection of glaucoma progression by assessment of segmented macular thickness data obtained using spectral domain optical coherence tomography. , 2012, Investigative ophthalmology & visual science.

[76]  Taimur Hassan,et al.  Deep structure tensor graph search framework for automated extraction and characterization of retinal layers and fluid pathology in retinal SD-OCT scans , 2019, Comput. Biol. Medicine.

[77]  H. Kim,et al.  Detection of glaucomatous progression by spectral-domain optical coherence tomography. , 2013, Ophthalmology.

[78]  Felipe A. Medeiros,et al.  A Review of Deep Learning for Screening, Diagnosis, and Detection of Glaucoma Progression , 2020, Translational vision science & technology.

[79]  R. P. Mills,et al.  Categorizing the stage of glaucoma from pre-diagnosis to end-stage disease. , 2006, American journal of ophthalmology-glaucoma.

[80]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[81]  Alexander Ilin,et al.  Semi-Supervised Few-Shot Learning with Prototypical Networks , 2017, ArXiv.

[82]  R. Ritch,et al.  Glaucoma – Authors' reply , 2018, The Lancet.

[83]  Sertan Serte,et al.  Transfer Learning for Early and Advanced Glaucoma Detection with Convolutional Neural Networks , 2019, 2019 Medical Technologies Congress (TIPTEKNO).

[84]  Hiroshi Murata,et al.  Using Deep Learning and Transfer Learning to Accurately Diagnose Early-Onset Glaucoma From Macular Optical Coherence Tomography Images. , 2019, American journal of ophthalmology.