ERICA: Emulated Retinal Image CApture - A tool for testing, training and validating retinal image processing methods

High resolution retinal imaging systems, such as adaptive optics scanning laser ophthalmoscopes (AOSLO), are increasingly being used for clinical and fundamental studies in neuroscience. These systems offer unprecedented spatial and temporal resolution of retinal structures in vivo. However, a major challenge is the development of robust and automated methods for processing and analysing these images. We present ERICA (Emulated Retinal Image CApture), a simulation tool that generates realistic synthetic images of the human cone mosaic, mimicking images that would be captured by an AOSLO, with specified image quality and with corresponding ground truth data. The simulation includes a self-organising mosaic of photoreceptors, the eye movements an observer might make during image capture, and data capture through a real system incorporating diffraction, residual optical aberrations and noise. The retinal photoreceptor mosaics generated by ERICA have a similar packing geometry to human retina, as determined by expert labelling of AOSLO images of real eyes. In the current implementation ERICA outputs convincingly realistic en face images of the cone photoreceptor mosaic but extensions to other imaging modalities and structures are also discussed. These images and associated ground-truth data can be used to develop, test and validate image processing and analysis algorithms or to train and validate machine learning approaches. The use of synthetic images has the advantage that neither access to an imaging system, nor to human participants is necessary for development.

[1]  Sumitha L. Balasuriya,et al.  A biologically inspired computational vision front-end based on a self-organised pseudo-randomly tessellated artificial retina , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[2]  A. Roorda,et al.  Adaptive optics ophthalmoscopy. , 2015, Annual review of vision science.

[3]  Ian J. C. MacCormick,et al.  Developing retinal biomarkers of neurological disease: an analytical perspective , 2015, Biomarkers in medicine.

[4]  Donald T. Miller,et al.  In vivo functional imaging of human cone photoreceptors. , 2007, Optics express.

[5]  Emily J Patterson,et al.  Adaptive optics imaging of inherited retinal diseases , 2017, British Journal of Ophthalmology.

[6]  C. K. Sheehy,et al.  Active eye-tracking for an adaptive optics scanning laser ophthalmoscope. , 2015, Biomedical optics express.

[7]  R. Webb Confocal optical microscopy , 1996 .

[8]  Austin Roorda,et al.  High-speed, image-based eye tracking with a scanning laser ophthalmoscope , 2012, Biomedical optics express.

[9]  Nicholas Devaney,et al.  Pre‐processing, registration and selection of adaptive optics corrected retinal images , 2013, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[10]  Austin Roorda,et al.  Mapping the Perceptual Grain of the Human Retina , 2014, The Journal of Neuroscience.

[11]  Heather E Moss,et al.  Retinal Vascular Changes are a Marker for Cerebral Vascular Diseases , 2015, Current Neurology and Neuroscience Reports.

[12]  Sina Farsiu,et al.  Open source software for automatic detection of cone photoreceptors in adaptive optics ophthalmoscopy using convolutional neural networks , 2017, Scientific Reports.

[13]  Hannah E. Smithson,et al.  What makes a microsaccade? A review of 70 years of research prompts a new detection method. , 2020, Journal of eye movement research.

[14]  Michael Smith,et al.  An Analytical Perspective , 2005 .

[15]  Jessica C. Hsu,et al.  The Reliability of Cone Density Measurements in the Presence of Rods , 2018, Translational vision science & technology.

[16]  Jessica I. W. Morgan,et al.  Optoretinography of individual human cone photoreceptors. , 2020, Optics express.

[17]  C W Tyler,et al.  Analysis of visual modulation sensitivity. II. Peripheral retina and the role of photoreceptor dimensions. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[18]  Austin Roorda,et al.  Automated identification of cone photoreceptors in adaptive optics retinal images. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  Travis E. Oliphant,et al.  Guide to NumPy , 2015 .

[20]  Austin Roorda,et al.  Retinally stabilized cone-targeted stimulus delivery. , 2007, Optics express.

[21]  Phillip Bedggood,et al.  De-warping of images and improved eye tracking for the scanning laser ophthalmoscope , 2017, PloS one.

[22]  R. Navarro,et al.  Odd aberrations and double-pass measurements of retinal image quality. , 1995, Journal of the Optical Society of America. A, Optics, image science, and vision.

[23]  Christopher S. Langlo,et al.  Assessing Photoreceptor Structure in Retinitis Pigmentosa and Usher Syndrome , 2016, Investigative ophthalmology & visual science.

[24]  D. Williams,et al.  Cone spacing and the visual resolution limit. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[25]  Hannah E. Smithson,et al.  Compact, modular and in-plane AOSLO for high-resolution retinal imaging , 2018, Biomedical optics express.

[26]  D R Williams,et al.  Supernormal vision and high-resolution retinal imaging through adaptive optics. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[27]  Jennifer J. Hunter,et al.  Vision science and adaptive optics, the state of the field , 2017, Vision Research.

[28]  Peter Dirksen,et al.  Assessment of an extended Nijboer-Zernike approach for the computation of optical point-spread functions. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[29]  S. Burns,et al.  In vivo measurement of erythrocyte velocity and retinal blood flow using adaptive optics scanning laser ophthalmoscopy. , 2008, Optics express.

[30]  Mohammed Azmi Al-Betar,et al.  comprehensive review : Krill Herd algorithm ( KH ) and its pplications saju , 2016 .

[31]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[32]  Alfredo Dubra,et al.  Registration of 2D Images from Fast Scanning Ophthalmic Instruments , 2010, WBIR.

[33]  M. Rucci,et al.  Precision of sustained fixation in trained and untrained observers. , 2012, Journal of vision.

[34]  Martina Poletti,et al.  Control and Functions of Fixational Eye Movements. , 2015, Annual review of vision science.

[35]  Christopher S. Langlo,et al.  Repeatability of In Vivo Parafoveal Cone Density and Spacing Measurements , 2012, Optometry and vision science : official publication of the American Academy of Optometry.

[36]  T. Hebert,et al.  Adaptive optics scanning laser ophthalmoscopy. , 2002, Optics express.

[37]  Austin Roorda,et al.  Adaptive optics for studying visual function: a comprehensive review. , 2011, Journal of vision.

[38]  Girish Kumar,et al.  Characteristics of fixational eye movements in people with macular disease. , 2014, Investigative ophthalmology & visual science.

[39]  Austin Roorda,et al.  The effects of fixational tremor on the retinal image , 2019, Journal of vision.

[40]  Scott B Stevenson,et al.  Psychophysical measurements of referenced and unreferenced motion processing using high-resolution retinal imaging. , 2008, Journal of vision.

[41]  Zhuolin Liu,et al.  Modal content of living human cone photoreceptors. , 2015, Biomedical optics express.

[42]  David Williams,et al.  The arrangement of the three cone classes in the living human eye , 1999, Nature.

[43]  Rashid Ansari,et al.  Frequency-based local content adaptive filtering algorithm for automated photoreceptor cell density quantification , 2012, 2012 19th IEEE International Conference on Image Processing.

[44]  Serge Meimon,et al.  High temporal resolution aberrometry in a 50-eye population and implications for adaptive optics error budget. , 2017, Biomedical optics express.

[45]  Austin Roorda,et al.  Miniature eye movements measured simultaneously with ophthalmic imaging and a dual-Purkinje image eye tracker , 2010 .

[46]  Benedikt V Ehinger,et al.  Probing the temporal dynamics of the exploration-exploitation dilemma of eye movements. , 2018, Journal of vision.

[47]  Sebastien Ourselin,et al.  Automatic Cone Photoreceptor Localisation in Healthy and Stargardt Afflicted Retinas Using Deep Learning , 2018, Scientific Reports.

[48]  Austin Roorda,et al.  Benefits of retinal image motion at the limits of spatial vision , 2017, Journal of vision.

[49]  Toco Y P Chui,et al.  Adaptive-optics imaging of human cone photoreceptor distribution. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[50]  David H Brainard,et al.  Multi-modal automatic montaging of adaptive optics retinal images. , 2016, Biomedical optics express.

[51]  N. Cottaris,et al.  A computational-observer model of spatial contrast sensitivity: Effects of wave-front-based optics, cone-mosaic structure, and inference engine. , 2019, Journal of vision.

[52]  Lawrence C. Sincich,et al.  Light reflectivity and interference in cone photoreceptors. , 2019, Biomedical optics express.

[53]  Nicusor Iftimia,et al.  Compact adaptive optics line scanning ophthalmoscope. , 2009, Optics express.

[54]  A. Roorda,et al.  Intrinsic signals from human cone photoreceptors. , 2008, Investigative ophthalmology & visual science.

[55]  Ahmadreza Baghaie,et al.  An Automated Reference Frame Selection (ARFS) Algorithm for Cone Imaging with Adaptive Optics Scanning Light Ophthalmoscopy , 2017, Translational vision science & technology.

[56]  anonymous,et al.  Comprehensive review , 2019 .

[57]  Philip J. Morrow,et al.  Automated Identification of Photoreceptor Cones Using Multi-scale Modelling and Normalized Cross-Correlation , 2011, ICIAP.

[58]  Austin Roorda,et al.  Speed quantification and tracking of moving objects in adaptive optics scanning laser ophthalmoscopy. , 2011, Journal of biomedical optics.

[59]  Alfredo Dubra,et al.  Effects of Intraframe Distortion on Measures of Cone Mosaic Geometry from Adaptive Optics Scanning Light Ophthalmoscopy , 2016, Translational vision science & technology.

[60]  N. Cottaris,et al.  A computational observer model of spatial contrast sensitivity: Effects of wavefront-based optics, cone mosaic structure, and inference engine , 2018, bioRxiv.

[61]  G. M. Morris,et al.  Images of cone photoreceptors in the living human eye , 1996, Vision Research.

[62]  A. Turing The chemical basis of morphogenesis , 1952, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.

[63]  P. E. Hallett,et al.  Power spectra for ocular drift and tremor , 1985, Vision Research.

[64]  A. Elsner,et al.  Adaptive optics imaging of the human retina , 2019, Progress in Retinal and Eye Research.

[65]  Alfredo Dubra,et al.  Adaptive Optics Retinal Imaging – Clinical Opportunities and Challenges , 2013, Current eye research.

[66]  J. Enoch Optical Properties of the Retinal Receptors , 1963 .

[67]  Jessica I W Morgan,et al.  The fundus photo has met its match: optical coherence tomography and adaptive optics ophthalmoscopy are here to stay , 2016, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[68]  Sina Farsiu,et al.  Automatic detection of cone photoreceptors in split detector adaptive optics scanning light ophthalmoscope images , 2016, Biomedical optics express.

[69]  Brendan Horton,et al.  An analytical perspective , 1996, Nature.

[70]  Austin Roorda,et al.  Adaptive optics retinal imaging: emerging clinical applications. , 2010, Optometry and vision science : official publication of the American Academy of Optometry.

[71]  D. Hubel,et al.  The role of fixational eye movements in visual perception , 2004, Nature Reviews Neuroscience.

[72]  Joseph A. Izatt,et al.  Automatic cone photoreceptor segmentation using graph theory and dynamic programming , 2013, Biomedical optics express.

[73]  Austin Roorda,et al.  Spatial summation in the human fovea: Do normal optical aberrations and fixational eye movements have an effect? , 2018, Journal of vision.

[74]  Lynn W. Sun,et al.  Multimodal Imaging of Photoreceptor Structure in Choroideremia , 2016, PloS one.

[75]  Joseph Carroll,et al.  Methods for investigating the local spatial anisotropy and the preferred orientation of cones in adaptive optics retinal images , 2016, Visual Neuroscience.

[76]  David Williams,et al.  Adaptive optics retinal imaging in the living mouse eye , 2012, Biomedical optics express.

[77]  Jing Wang,et al.  Quality improvement of adaptive optics retinal images using conditional adversarial networks. , 2020, Biomedical optics express.

[78]  Nicholas Devaney,et al.  Performance Analysis of Cone Detection Algorithms , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[79]  A. Janssen Extended Nijboer-Zernike approach for the computation of optical point-spread functions. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[80]  Yudong Zhang,et al.  Automatic Dewarping of Retina Images in Adaptive Optics Confocal Scanning Laser Ophthalmoscope , 2019, IEEE Access.

[81]  A. Hendrickson,et al.  Human photoreceptor topography , 1990, The Journal of comparative neurology.

[82]  J. E. Pearson Complex Patterns in a Simple System , 1993, Science.

[83]  Austin Roorda,et al.  Correcting for miniature eye movements in high resolution scanning laser ophthalmoscopy , 2005 .

[84]  Austin Roorda,et al.  Design of an integrated hardware interface for AOSLO image capture and cone-targeted stimulus delivery , 2010, Optics express.

[85]  A. Roorda,et al.  Theoretical modeling and evaluation of the axial resolution of the adaptive optics scanning laser ophthalmoscope. , 2004, Journal of biomedical optics.

[86]  Austin Roorda,et al.  Suboptimal eye movements for seeing fine details , 2017, bioRxiv.

[87]  Sina Farsiu,et al.  Deep learning based detection of cone photoreceptors with multimodal adaptive optics scanning light ophthalmoscope images of achromatopsia. , 2018, Biomedical optics express.

[88]  Benedikt V Ehinger,et al.  A new comprehensive eye-tracking test battery concurrently evaluating the Pupil Labs glasses and the EyeLink 1000 , 2019, PeerJ.

[89]  John S Werner,et al.  Photoreceptor counting and montaging of en-face retinal images from an adaptive optics fundus camera. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[90]  A. Watson A formula for human retinal ganglion cell receptive field density as a function of visual field location. , 2014, Journal of vision.

[91]  B. Sullenger,et al.  Emerging clinical applications of RNA , 2002, Nature.