High-acuity vision from retinal image motion

A mathematical model and a possible neural mechanism are proposed to account for how fixational drift motion in the retina confers a benefit for the discrimination of high-acuity targets. We show that by simultaneously estimating object shape and eye motion, neurons in visual cortex can compute a higher quality representation of an object by averaging out non-uniformities in the retinal sampling lattice. The model proposes that this is accomplished by two separate populations of cortical neurons — one providing a representation of object shape and another representing eye position or motion — which are coupled through specific multiplicative connections. Combined with recent experimental findings, our model suggests that the visual system may utilize principles not unlike those used in computational imaging for achieving “super-resolution” via camera motion.

[1]  B. C. Motter,et al.  Dynamic stabilization of receptive fields of cortical neurons (VI) during fixation of gaze in the macaque , 2004, Experimental Brain Research.

[2]  Martina Poletti,et al.  Miniature eye movements enhance fine spatial detail , 2007, Nature.

[3]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[4]  D. V. van Essen,et al.  The representation of the visual field in parvicellular and magnocellular layers of the lateral geniculate nucleus in the macaque monkey , 1984, The Journal of comparative neurology.

[5]  David Williams,et al.  A visual nonlinearity fed by single cones , 1992, Vision Research.

[6]  T. Hebert,et al.  Adaptive optics scanning laser ophthalmoscopy. , 2002, Optics express.

[7]  Austin Roorda,et al.  Benefits of retinal image motion at the limits of spatial vision , 2017, Journal of vision.

[8]  H. Sompolinsky,et al.  A Neural Computation for Visual Acuity in the Presence of Eye Movements , 2007, PLoS biology.

[9]  Lawrence C. Sincich,et al.  Resolving Single Cone Inputs to Visual Receptive Fields , 2009, Nature Neuroscience.

[10]  Charles Elkan,et al.  Expectation Maximization Algorithm , 2010, Encyclopedia of Machine Learning.

[11]  Hamutal Slovin,et al.  Spatiotemporal effects of microsaccades on population activity in the visual cortex of monkeys during fixation. , 2012, Cerebral cortex.

[12]  Michele Rucci,et al.  Contrast sensitivity reveals an oculomotor strategy for temporally encoding space , 2018, bioRxiv.

[13]  Austin Roorda,et al.  Eye tracking with the adaptive optics scanning laser ophthalmoscope , 2010, ETRA.

[14]  B. B. Lee,et al.  Steady discharges of macaque retinal ganglion cells , 1991, Visual Neuroscience.

[15]  Kareem M. Ahmad,et al.  Cell density ratios in a foveal patch in macaque retina , 2003, Visual Neuroscience.

[16]  I. Donaldson,et al.  The functions of the proprioceptors of the eye muscles. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[17]  Bruce G. Cumming,et al.  High-resolution eye tracking using V1 neuron activity , 2014, Nature Communications.

[18]  Eero P. Simoncelli,et al.  Maximum Likelihood Estimation of a Stochastic Integrate-and-Fire Neural Model , 2003, NIPS.

[19]  Scott B Stevenson,et al.  How the unstable eye sees a stable and moving world. , 2013, Journal of vision.

[20]  J. V. van Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[21]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[22]  A. Roorda,et al.  Visual Psychophysics and Physiological Optics Relationship Between Foveal Cone Structure and Clinical Measures of Visual Function in Patients With Inherited Retinal Degenerations , 2013 .

[23]  Patrick Cavanagh,et al.  Visual jitter: evidence for visual-motion-based compensation of retinal slip due to small eye movements , 2001, Vision Research.

[24]  Richard G. Baraniuk,et al.  Sparse Coding via Thresholding and Local Competition in Neural Circuits , 2008, Neural Computation.

[25]  J. Victor,et al.  Temporal Encoding of Spatial Information during Active Visual Fixation , 2012, Current Biology.

[26]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[27]  J. Victor,et al.  The unsteady eye: an information-processing stage, not a bug , 2015, Trends in Neurosciences.

[28]  D C Van Essen,et al.  Shifter circuits: a computational strategy for dynamic aspects of visual processing. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[29]  William S Tuten,et al.  The elementary representation of spatial and color vision in the human retina , 2016, Science Advances.

[30]  Heidi Hofer,et al.  Organization of the Human Trichromatic Cone Mosaic , 2003, The Journal of Neuroscience.

[31]  M. Lenzen,et al.  Scientists’ warning on affluence , 2020, Nature Communications.

[32]  Moshe Gur PII: S0042-6989(96)00182-4 , 1997 .

[33]  Marco Boi,et al.  Consequences of the Oculomotor Cycle for the Dynamics of Perception , 2017, Current Biology.

[34]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[35]  Keith Mathieson,et al.  Retinal Representation of the Elementary Visual Signal , 2014, Neuron.

[36]  Austin Roorda,et al.  Mapping the Perceptual Grain of the Human Retina , 2014, The Journal of Neuroscience.

[37]  Haim Sompolinsky,et al.  Bayesian model of dynamic image stabilization in the visual system , 2010, Proceedings of the National Academy of Sciences.

[38]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[39]  Michele Rucci,et al.  Finely tuned eye movements enhance visual acuity , 2020, Nature Communications.

[40]  Michele Rucci,et al.  The Visual Input to the Retina during Natural Head-Free Fixation , 2014, The Journal of Neuroscience.

[41]  A. Swaroop,et al.  High-resolution imaging with adaptive optics in patients with inherited retinal degeneration. , 2007, Investigative ophthalmology & visual science.

[42]  D. Burr,et al.  Temporal Coding of Visual Space , 2018, Trends in Cognitive Sciences.

[43]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[44]  D. Sparks,et al.  Corollary discharge provides accurate eye position information to the oculomotor system. , 1983, Science.

[45]  D. Snodderly,et al.  Selective activation of visual cortex neurons by fixational eye movements: Implications for neural coding , 2001, Visual Neuroscience.