From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction

Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess the degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.

[1]  T H Bullock,et al.  Event-related potentials in the retina and optic tectum of fish. , 1990, Journal of neurophysiology.

[2]  Michael J. Berry,et al.  Sophisticated temporal pattern recognition in retinal ganglion cells. , 2008, Journal of neurophysiology.

[3]  Surya Ganguli,et al.  The dynamic neural code of the retina for natural scenes , 2019 .

[4]  Leon A. Gatys,et al.  Deep convolutional models improve predictions of macaque V1 responses to natural images , 2019, PLoS Comput. Biol..

[5]  Nikhil Rajiv Deshmukh,et al.  Complex computation in the retina , 2015 .

[6]  Surya Ganguli,et al.  Universality and individuality in neural dynamics across large populations of recurrent networks , 2019, NeurIPS.

[7]  Konrad P. Körding,et al.  The roles of supervised machine learning in systems neuroscience , 2018, Progress in Neurobiology.

[8]  T H Bullock,et al.  Dynamic properties of human visual evoked and omitted stimulus potentials. , 1994, Electroencephalography and clinical neurophysiology.

[9]  James J DiCarlo,et al.  Neural population control via deep image synthesis , 2018, Science.

[10]  Tim Gollisch,et al.  Rapid Neural Coding in the Retina with Relative Spike Latencies , 2008, Science.

[11]  Michael J. Berry,et al.  Anticipation of moving stimuli by the retina , 1999, Nature.

[12]  Michael J. Berry,et al.  Synchronized Firing among Retinal Ganglion Cells Signals Motion Reversal , 2007, Neuron.

[13]  Marcel A. J. van Gerven,et al.  Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream , 2014, The Journal of Neuroscience.

[14]  Alexander S. Ecker,et al.  Inception in visual cortex: in vivo-silico loops reveal most exciting images , 2018 .

[15]  L. Shapley,et al.  Values of Non-Atomic Games , 1974 .

[16]  Ankur Taly,et al.  Axiomatic Attribution for Deep Networks , 2017, ICML.

[17]  Michael J. Berry,et al.  Alert Response to Motion Onset in the Retina , 2013, The Journal of Neuroscience.

[18]  Michael J. Berry,et al.  Detection and prediction of periodic patterns by the retina , 2007, Nature Neuroscience.

[19]  Michael J. Berry,et al.  An oscillatory circuit underlying the detection of disruptions in temporally-periodic patterns , 2009, Network.

[20]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[21]  Michael J. Berry,et al.  The Neural Circuit Mechanisms Underlying the Retinal Response to Motion Reversal , 2014, The Journal of Neuroscience.

[22]  E. Marder,et al.  Understanding circuit dynamics using the stomatogastric nervous system of lobsters and crabs. , 2007, Annual review of physiology.

[23]  Mukund Sundararajan,et al.  How Important Is a Neuron? , 2018, ICLR.

[24]  Benjamin Sivyer,et al.  Direction selectivity in the retina: symmetry and asymmetry in structure and function , 2012, Nature Reviews Neuroscience.

[25]  Tim Gollisch,et al.  Modeling convergent ON and OFF pathways in the early visual system , 2008, Biological Cybernetics.

[26]  Surya Ganguli,et al.  Deep Learning Models of the Retinal Response to Natural Scenes , 2017, NIPS.

[27]  Christopher L Passaglia,et al.  Complex temporal response patterns with a simple retinal circuit. , 2008, Journal of neurophysiology.

[28]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[29]  Tim Gollisch,et al.  Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina , 2010, Neuron.

[30]  M. Konishi Coding of auditory space. , 2003, Annual review of neuroscience.

[31]  Jakob H. Macke,et al.  Analyzing biological and artificial neural networks: challenges with opportunities for synergy? , 2018, Current Opinion in Neurobiology.

[32]  Surya Ganguli,et al.  Deep learning models reveal internal structure and diverse computations in the retina under natural scenes , 2018, bioRxiv.

[33]  Alexander S. Ecker,et al.  Inception loops discover what excites neurons most using deep predictive models , 2019, Nature Neuroscience.

[34]  Leon A. Gatys,et al.  Deep convolutional models improve predictions of macaque V1 responses to natural images , 2017, bioRxiv.