RAC-CNN: multimodal deep learning based automatic detection and classification of rod and cone photoreceptors in adaptive optics scanning light ophthalmoscope images.

Quantification of the human rod and cone photoreceptor mosaic in adaptive optics scanning light ophthalmoscope (AOSLO) images is useful for the study of various retinal pathologies. Subjective and time-consuming manual grading has remained the gold standard for evaluating these images, with no well validated automatic methods for detecting individual rods having been developed. We present a novel deep learning based automatic method, called the rod and cone CNN (RAC-CNN), for detecting and classifying rods and cones in multimodal AOSLO images. We test our method on images from healthy subjects as well as subjects with achromatopsia over a range of retinal eccentricities. We show that our method is on par with human grading for detecting rods and cones.

[1]  A. Roorda,et al.  Adaptive optics ophthalmoscopy. , 2015, Annual review of vision science.

[2]  Ashavini M. Pavaskar,et al.  Spatial and temporal variation of rod photoreceptor reflectance in the human retina , 2011, Biomedical optics express.

[3]  M. Abràmoff,et al.  Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning. , 2016, Investigative ophthalmology & visual science.

[4]  Fred K Chen,et al.  Semi-automated identification of cones in the human retina using circle Hough transform. , 2015, Biomedical optics express.

[5]  Jennifer J. Hunter,et al.  Imaging individual neurons in the retinal ganglion cell layer of the living eye , 2017, Proceedings of the National Academy of Sciences.

[6]  Krzysztof Krawiec,et al.  Segmenting Retinal Blood Vessels With Deep Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[7]  Joseph A. Izatt,et al.  Automatic segmentation of closed-contour features in ophthalmic images using graph theory and dynamic programming , 2012, Biomedical optics express.

[8]  Christopher S. Langlo,et al.  In vivo imaging of human cone photoreceptor inner segments. , 2014, Investigative ophthalmology & visual science.

[9]  A. Dubra,et al.  Subclinical photoreceptor disruption in response to severe head trauma. , 2012, Archives of ophthalmology.

[10]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[11]  Yudong Zhang,et al.  Deblurring adaptive optics retinal images using deep convolutional neural networks. , 2017, Biomedical optics express.

[12]  Sebastien Ourselin,et al.  Automatic Cone Photoreceptor Localisation in Healthy and Stargardt Afflicted Retinas Using Deep Learning , 2018, Scientific Reports.

[13]  Sina Farsiu,et al.  Deep longitudinal transfer learning-based automatic segmentation of photoreceptor ellipsoid zone defects on optical coherence tomography images of macular telangiectasia type 2 , 2018, Biomedical optics express.

[14]  A. Dubra,et al.  Sub-Airy Confocal Adaptive Optics Scanning Ophthalmoscopy , 2018, Translational vision science & technology.

[15]  M. Lombardo,et al.  Technical Factors Influencing Cone Packing Density Estimates in Adaptive Optics Flood Illuminated Retinal Images , 2014, PloS one.

[16]  Jungtae Rha,et al.  Imaging the photoreceptor mosaic with adaptive optics: beyond counting cones. , 2012, Advances in experimental medicine and biology.

[17]  Sina Farsiu,et al.  Fast and robust active neuron segmentation in two-photon calcium imaging using spatiotemporal deep learning , 2019, Proceedings of the National Academy of Sciences.

[18]  Bing Wu,et al.  Automated analysis of differential interference contrast microscopy images of the foveal cone mosaic. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  S. Ourselin,et al.  Unsupervised identification of cone photoreceptors in non-confocal adaptive optics scanning light ophthalmoscope images. , 2017, Biomedical optics express.

[20]  Joseph A. Izatt,et al.  Automatic cone photoreceptor segmentation using graph theory and dynamic programming , 2013, Biomedical optics express.

[21]  A. Dubra,et al.  Reflective afocal broadband adaptive optics scanning ophthalmoscope , 2011, Biomedical optics express.

[22]  A. Elsner,et al.  Adaptive optics imaging of the human retina , 2019, Progress in Retinal and Eye Research.

[23]  Ravi S. Jonnal,et al.  Imaging cone photoreceptors in three dimensions and in time using ultrahigh resolution optical coherence tomography with adaptive optics , 2011, Biomedical optics express.

[24]  Steven M. Jones,et al.  Adaptive-optics optical coherence tomography for high-resolution and high-speed 3D retinal in vivo imaging. , 2005, Optics express.

[25]  Debjani Chakraborty,et al.  Transfer learning based classification of optical coherence tomography images with diabetic macular edema and dry age-related macular degeneration. , 2017, Biomedical optics express.

[26]  Christopher S. Langlo,et al.  Repeatability of In Vivo Parafoveal Cone Density and Spacing Measurements , 2012, Optometry and vision science : official publication of the American Academy of Optometry.

[27]  David R Williams,et al.  Cone and rod loss in Stargardt disease revealed by adaptive optics scanning light ophthalmoscopy. , 2015, JAMA ophthalmology.

[28]  A. Roorda,et al.  Observation of cone and rod photoreceptors in normal subjects and patients using a new generation adaptive optics scanning laser ophthalmoscope , 2011, Biomedical optics express.

[29]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[30]  Joseph A. Izatt,et al.  Handheld Adaptive Optics Scanning Laser Ophthalmoscope. , 2018, Optica.

[31]  David Williams,et al.  Noninvasive imaging of the human rod photoreceptor mosaic using a confocal adaptive optics scanning ophthalmoscope , 2011, Biomedical optics express.

[32]  Donald T. Miller,et al.  Imaging and quantifying ganglion cells and other transparent neurons in the living human retina , 2017, Proceedings of the National Academy of Sciences.

[33]  S. Burns,et al.  Enhanced retinal vasculature imaging with a rapidly configurable aperture. , 2018, Biomedical optics express.

[34]  Myeong Jin Ju,et al.  Automated identification of cone photoreceptors in adaptive optics optical coherence tomography images using transfer learning. , 2018, Biomedical optics express.

[35]  Christopher S. Langlo,et al.  Residual Foveal Cone Structure in CNGB3-Associated Achromatopsia , 2016, Investigative ophthalmology & visual science.

[36]  W. Drexler,et al.  Adaptive optics optical coherence tomography at 120,000 depth scans/s for non-invasive cellular phenotyping of the living human retina. , 2009, Optics express.

[37]  Sina Farsiu,et al.  Deep learning based detection of cone photoreceptors with multimodal adaptive optics scanning light ophthalmoscope images of achromatopsia. , 2018, Biomedical optics express.

[38]  Sina Farsiu,et al.  Open-source, machine and deep learning-based automated algorithm for gestational age estimation through smartphone lens imaging. , 2018, Biomedical optics express.

[39]  Sina Farsiu,et al.  Enhanced visualization of peripheral retinal vasculature with wavefront sensorless adaptive optics optical coherence tomography angiography in diabetic patients. , 2017, Optics letters.

[40]  A. Hendrickson,et al.  Human photoreceptor topography , 1990, The Journal of comparative neurology.

[41]  Sina Farsiu,et al.  Super-resolution retinal imaging using optically reassigned scanning laser ophthalmoscopy , 2019, Nature photonics.

[42]  Sina Farsiu,et al.  Automatic detection of cone photoreceptors in split detector adaptive optics scanning light ophthalmoscope images , 2016, Biomedical optics express.

[43]  Christopher S. Langlo,et al.  Automatic detection of modal spacing (Yellott's ring) in adaptive optics scanning light ophthalmoscope images , 2013, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[44]  Jianfei Liu,et al.  Automated Photoreceptor Cell Identification on Nonconfocal Adaptive Optics Images Using Multiscale Circular Voting , 2017, Investigative ophthalmology & visual science.

[45]  Jennifer J. Hunter,et al.  Two-Photon Autofluorescence Imaging Reveals Cellular Structures Throughout the Retina of the Living Primate Eye , 2016, Investigative ophthalmology & visual science.

[46]  Austin Roorda,et al.  Automated identification of cone photoreceptors in adaptive optics retinal images. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[47]  David Williams,et al.  The arrangement of the three cone classes in the living human eye , 1999, Nature.

[48]  Christopher S. Langlo,et al.  Reliability and Repeatability of Cone Density Measurements in Patients with Congenital Achromatopsia. , 2016, Advances in experimental medicine and biology.

[49]  Nicholas Devaney,et al.  Performance Analysis of Cone Detection Algorithms , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[50]  T. Hebert,et al.  Adaptive optics scanning laser ophthalmoscopy. , 2002, Optics express.

[51]  John S Werner,et al.  Photoreceptor counting and montaging of en-face retinal images from an adaptive optics fundus camera. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[52]  P. Artal,et al.  Adaptive-optics ultrahigh-resolution optical coherence tomography. , 2004, Optics letters.

[53]  Nassir Navab,et al.  ReLayNet: Retinal Layer and Fluid Segmentation of Macular Optical Coherence Tomography using Fully Convolutional Network , 2017, Biomedical optics express.

[54]  David R. Williams,et al.  Images of photoreceptors in living primate eyes using adaptive optics two-photon ophthalmoscopy , 2010, Biomedical optics express.

[55]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[56]  Jennifer J. Hunter,et al.  Adaptive optics two-photon excited fluorescence lifetime imaging ophthalmoscopy of exogenous fluorophores in mice. , 2017, Biomedical optics express.

[57]  A. Dubra,et al.  In vivo imaging of human retinal microvasculature using adaptive optics scanning light ophthalmoscope fluorescein angiography , 2013, Biomedical optics express.

[58]  Sina Farsiu,et al.  Open source software for automatic detection of cone photoreceptors in adaptive optics ophthalmoscopy using convolutional neural networks , 2017, Scientific Reports.

[59]  D R Williams,et al.  Supernormal vision and high-resolution retinal imaging through adaptive optics. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[60]  Tianfu Wang,et al.  A Cross-Modality Learning Approach for Vessel Segmentation in Retinal Images , 2016, IEEE Transactions on Medical Imaging.

[61]  Yifan Jian,et al.  Wavefront sensorless adaptive optics optical coherence tomography for in vivo retinal imaging in mice. , 2014, Biomedical optics express.

[62]  Bram van Ginneken,et al.  Fast Convolutional Neural Network Training Using Selective Data Sampling: Application to Hemorrhage Detection in Color Fundus Images , 2016, IEEE Transactions on Medical Imaging.

[63]  Bernard P. Gee,et al.  In vivo fluorescence imaging of primate retinal ganglion cells and retinal pigment epithelial cells. , 2006, Optics express.

[64]  Chong Wang,et al.  Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. , 2017, Biomedical optics express.