Natural images are reliably represented by sparse and variable populations of neurons in visual cortex

Natural scenes sparsely activate neurons in the primary visual cortex (V1). However, how sparsely active neurons reliably represent complex natural images and how the information is optimally decoded from these representations have not been revealed. Using two-photon calcium imaging, we recorded visual responses to natural images from several hundred V1 neurons and reconstructed the images from neural activity in anesthetized and awake mice. A single natural image is linearly decodable from a surprisingly small number of highly responsive neurons, and the remaining neurons even degrade the decoding. Furthermore, these neurons reliably represent the image across trials, regardless of trial-to-trial response variability. Based on our results, diverse, partially overlapping receptive fields ensure sparse and reliable representation. We suggest that information is reliably represented while the corresponding neuronal patterns change across trials and collecting only the activity of highly responsive neurons is an optimal decoding strategy for the downstream neurons. Natural scenes sparsely activate V1 neurons. Here, the authors show that a small number of active cells reliably represent visual contents of a natural image across trials regardless of response variability, due to the diverse and partially overlapping representations of individual cells.

[1]  J. Gallant,et al.  Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies , 2011, Current Biology.

[2]  Martin Rehn,et al.  A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields , 2007, Journal of Computational Neuroscience.

[3]  David J. Field,et al.  How Close Are We to Understanding V1? , 2005, Neural Computation.

[4]  József Fiser,et al.  Coding of Natural Scenes in Primary Visual Cortex , 2003, Neuron.

[5]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[6]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[7]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[8]  Bruno A. Olshausen,et al.  Learning real and complex overcomplete representations from the statistics of natural images , 2009, Optical Engineering + Applications.

[9]  Andriana Olmos,et al.  A biologically inspired algorithm for the recovery of shading and reflectance images , 2004 .

[10]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[11]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[12]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[13]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[14]  R. Reid,et al.  Local Diversity and Fine-Scale Organization of Receptive Fields in Mouse Visual Cortex , 2011, The Journal of Neuroscience.

[15]  Sooyoung Chung,et al.  Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex , 2005, Nature.

[16]  Brenda C. Shields,et al.  Thy1-GCaMP6 Transgenic Mice for Neuronal Population Imaging In Vivo , 2014, PloS one.

[17]  Jonathan Westley Peirce,et al.  Neuroinformatics Original Research Article Generating Stimuli for Neuroscience Using Psychopy , 2022 .

[18]  M. Carandini,et al.  Locomotion Controls Spatial Integration in Mouse Visual Cortex , 2013, Current Biology.

[19]  M. Cohen,et al.  Measuring and interpreting neuronal correlations , 2011, Nature Neuroscience.

[20]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[21]  Toby Berger,et al.  Reliable On-Line Human Signature Verification Systems , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Allan R. Jones,et al.  A robust and high-throughput Cre reporting and characterization system for the whole mouse brain , 2009, Nature Neuroscience.

[23]  Bruno A. Olshausen,et al.  Highly overcomplete sparse coding , 2013, Electronic Imaging.

[24]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[25]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[26]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[27]  Kenichi Ohki,et al.  Representation of natural image contents by sparsely active neurons in visual cortex , 2018 .

[28]  Edmund T. Rolls,et al.  What determines the capacity of autoassociative memories in the brain? Network , 1991 .

[29]  R. Segev,et al.  How silent is the brain: is there a “dark matter” problem in neuroscience? , 2006, Journal of Comparative Physiology A.

[30]  G B Stanley,et al.  Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus , 1999, The Journal of Neuroscience.

[31]  Mriganka Sur,et al.  Spatial Correlations in Natural Scenes Modulate Response Reliability in Mouse Visual Cortex , 2015, The Journal of Neuroscience.

[32]  S. Nelson,et al.  A Resource of Cre Driver Lines for Genetic Targeting of GABAergic Neurons in Cerebral Cortex , 2011, Neuron.

[33]  B. Willmore,et al.  Sparse coding in striate and extrastriate visual cortex. , 2011, Journal of neurophysiology.

[34]  Y Kamitani,et al.  Neural Decoding of Visual Imagery During Sleep , 2013, Science.

[35]  E T Rolls,et al.  Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. , 1995, Journal of neurophysiology.

[36]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[37]  P. J. Sjöström,et al.  Functional specificity of local synaptic connections in neocortical networks , 2011, Nature.

[38]  David S. Greenberg,et al.  Population imaging of ongoing neuronal activity in the visual cortex of awake rats , 2008, Nature Neuroscience.

[39]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.

[40]  Kenichi Ohki,et al.  Robust representation of natural images by sparse and variable population of active neurons in visual cortex , 2018 .

[41]  F. Helmchen,et al.  Sulforhodamine 101 as a specific marker of astroglia in the neocortex in vivo , 2004, Nature Methods.

[42]  Morgane M. Roth,et al.  Representation of visual scenes by local neuronal populations in layer 2/3 of mouse visual cortex , 2011, Front. Neural Circuits.

[43]  Nicholas A. Steinmetz,et al.  High-dimensional geometry of population responses in visual cortex , 2018, Nature.

[44]  R. Yuste,et al.  Visual stimuli recruit intrinsically generated cortical ensembles , 2014, Proceedings of the National Academy of Sciences.

[45]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[46]  James H. Marshel,et al.  Functional Specialization of Seven Mouse Visual Cortical Areas , 2011, Neuron.

[47]  Kenichi Ohki,et al.  Neuronal activity is not required for the initial formation and maturation of visual selectivity , 2015, Nature Neuroscience.

[48]  C. Clopath,et al.  The emergence of functional microcircuits in visual cortex , 2013, Nature.

[49]  Feng Qi Han,et al.  Rapid learning in cortical coding of visual scenes , 2007, Nature Neuroscience.

[50]  Stefan R. Pulver,et al.  Ultra-sensitive fluorescent proteins for imaging neuronal activity , 2013, Nature.

[51]  Spencer L. Smith,et al.  Parallel processing of visual space by neighboring neurons in mouse visual cortex , 2010, Nature Neuroscience.

[52]  D. R. Muir,et al.  Functional organization of excitatory synaptic strength in primary visual cortex , 2015, Nature.

[53]  Alexander S. Ecker,et al.  Population code in mouse V1 facilitates read-out of natural scenes through increased sparseness , 2014, Nature Neuroscience.

[54]  Tai Sing Lee,et al.  Large-scale two-photon imaging revealed super-sparse population codes in the V1 superficial layer of awake monkeys , 2018, eLife.

[55]  W. M. Keck,et al.  Highly Selective Receptive Fields in Mouse Visual Cortex , 2008, The Journal of Neuroscience.

[56]  Michael S. Lewicki,et al.  Sparse Coding of Natural Images Using an Overcomplete Set of Limited Capacity Units , 2004, NIPS.

[57]  C. Gray,et al.  Heterogeneity in the responses of adjacent neurons to natural stimuli in cat striate cortex. , 2007, Journal of neurophysiology.

[58]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.

[59]  M. Stryker,et al.  Modulation of Visual Responses by Behavioral State in Mouse Visual Cortex , 2010, Neuron.

[60]  A. Pouget,et al.  Information-limiting correlations , 2014, Nature Neuroscience.

[61]  A. Borst,et al.  A genetically encoded calcium indicator for chronic in vivo two-photon imaging , 2008, Nature Methods.

[62]  D. Tolhurst,et al.  The Sparseness of Neuronal Responses in Ferret Primary Visual Cortex , 2009, The Journal of Neuroscience.

[63]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[64]  R. Reid,et al.  Broadly Tuned Response Properties of Diverse Inhibitory Neuron Subtypes in Mouse Visual Cortex , 2010, Neuron.

[65]  J. S Stahl,et al.  A comparison of video and magnetic search coil recordings of mouse eye movements , 2000, Journal of Neuroscience Methods.

[66]  Ben Willmore,et al.  The Receptive-Field Organization of Simple Cells in Primary Visual Cortex of Ferrets under Natural Scene Stimulation , 2003, The Journal of Neuroscience.

[67]  Tai Sing Lee,et al.  Image Representation Using 2D Gabor Wavelets , 1996, IEEE Trans. Pattern Anal. Mach. Intell..