The dynamic neural code of the retina for natural scenes

Understanding how the visual system encodes natural scenes is a fundamental goal of sensory neuroscience. We show here that a three-layer network model predicts the retinal response to natural scenes with an accuracy nearing the fundamental limits of predictability. The model’s internal structure is interpretable, in that model units are highly correlated with interneurons recorded separately and not used to fit the model. We further show the ethological relevance to natural visual processing of a diverse set of phenomena of complex motion encoding, adaptation and predictive coding. Our analysis uncovers a fast timescale of visual processing that is inaccessible directly from experimental data, showing unexpectedly that ganglion cells signal in distinct modes by rapidly (< 0.1 s) switching their selectivity for direction of motion, orientation, location and the sign of intensity. A new approach that decomposes ganglion cell responses into the contribution of interneurons reveals how the latent effects of parallel retinal circuits generate the response to any possible stimulus. These results reveal extremely flexible and rapid dynamics of the retinal code for natural visual stimuli, explaining the need for a large set of interneuron pathways to generate the dynamic neural code for natural scenes.

[1]  Michael J. Berry,et al.  Synchronized Firing among Retinal Ganglion Cells Signals Motion Reversal , 2007, Neuron.

[2]  M. Hayhoe Vision and Action. , 2017, Annual review of vision science.

[3]  S. Martinez-Conde,et al.  Fixational eye movements across vertebrates: comparative dynamics, physiology, and perception. , 2008, Journal of vision.

[4]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[5]  M. Meister,et al.  Dynamic predictive coding by the retina , 2005, Nature.

[6]  Masahito Yamagata,et al.  SIDEKICK 2 DIRECTS FORMATION OF A RETINAL CIRCUIT THAT DETECTS DIFFERENTIAL MOTION , 2015, Nature.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Saskia E. J. de Vries,et al.  Retinal Ganglion Cells Can Rapidly Change Polarity from Off to On , 2007, PLoS Biology.

[9]  H. Barlow,et al.  Change of organization in the receptive fields of the cat's retina during dark adaptation , 1957, The Journal of physiology.

[10]  Kerry J. Kim,et al.  Temporal Contrast Adaptation in the Input and Output Signals of Salamander Retinal Ganglion Cells , 2001, The Journal of Neuroscience.

[11]  Stephen A. Baccus,et al.  Segregation of object and background motion in the retina , 2003, Nature.

[12]  Michael J. Berry,et al.  Adaptation of retinal processing to image contrast and spatial scale , 1997, Nature.

[13]  Eero P. Simoncelli,et al.  Spatio-temporal correlations and visual signalling in a complete neuronal population , 2008, Nature.

[14]  E J Chichilnisky,et al.  A simple white noise analysis of neuronal light responses , 2001, Network.

[15]  Vijay Balasubramanian,et al.  Natural Images from the Birthplace of the Human Eye , 2011, PloS one.

[16]  J. Diamond Inhibitory Interneurons in the Retina: Types, Circuitry, and Function. , 2017, Annual review of vision science.

[17]  Michael J. Berry,et al.  Anticipation of moving stimuli by the retina , 1999, Nature.

[18]  Surya Ganguli,et al.  From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction , 2019, NeurIPS.

[19]  R. Shapley,et al.  Linear and nonlinear spatial subunits in Y cat retinal ganglion cells. , 1976, The Journal of physiology.

[20]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[21]  Tim Gollisch,et al.  Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina , 2010, Neuron.

[22]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[23]  Michael J. Berry,et al.  Detection and prediction of periodic patterns by the retina , 2007, Nature Neuroscience.

[24]  Katherine I. Nagel,et al.  Temporal Processing and Adaptation in the Songbird Auditory Forebrain , 2006, Neuron.

[25]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[26]  A Kaneko,et al.  Receptive field organization of bipolar and amacrine cells in the goldfish retina , 1973, The Journal of physiology.

[27]  M. Meister,et al.  Fast and Slow Contrast Adaptation in Retinal Circuitry , 2002, Neuron.

[28]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[29]  S. Baccus,et al.  Coordinated dynamic encoding in the retina using opposing forms of plasticity , 2011, Nature Neuroscience.

[30]  Surya Ganguli,et al.  Deep Learning Models of the Retinal Response to Natural Scenes , 2017, NIPS.