The statistics of how natural images drive the responses of neurons

To model the responses of neurons in the early visual system, at least three basic components are required: a receptive field, a normalization term, and a specification of encoding noise. Here, we examine how the receptive field, the normalization factor, and the encoding noise affect the drive to model-neuron responses when stimulated with natural images. We show that when these components are modeled appropriately, the response drives elicited by natural stimuli are Gaussian-distributed and scale invariant, and very nearly maximize the sensitivity (d′) for natural-image discrimination. We discuss the statistical models of natural stimuli that can account for these response statistics, and we show how some commonly used modeling practices may distort these results. Finally, we show that normalization can equalize important properties of neural response across different stimulus types. Specifically, narrowband (stimulus- and feature-specific) normalization causes model neurons to yield Gaussian response-drive statistics when stimulated with natural stimuli, 1/f noise stimuli, and white-noise stimuli. The current work makes recommendations for best practices and lays a foundation, grounded in the response statistics to natural stimuli, upon which to build principled models of more complex visual tasks.

[1]  Martin J. Wainwright,et al.  Scale Mixtures of Gaussians and the Statistics of Natural Images , 1999, NIPS.

[2]  Jonathan W. Pillow,et al.  Spectral methods for neural characterization using generalized quadratic models , 2013, NIPS.

[3]  Johannes Burge,et al.  Linking Normative Models of Natural Tasks to Descriptive Models of Neural Response , 2017 .

[4]  Timothy A. Machado,et al.  Functional connectivity in the retina at the resolution of photoreceptors , 2010, Nature.

[5]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[6]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[7]  D. G. Albrecht,et al.  Spatial frequency selectivity of cells in macaque visual cortex , 1982, Vision Research.

[8]  D. Tolhurst,et al.  Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes , 2000, Vision Research.

[9]  Eero P. Simoncelli,et al.  Origin and Function of Tuning Diversity in Macaque Visual Cortex , 2015, Neuron.

[10]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[11]  J. Alonso,et al.  Adaptation to Stimulus Contrast and Correlations during Natural Visual Stimulation , 2007, Neuron.

[12]  Eero P. Simoncelli,et al.  Spike-triggered neural characterization. , 2006, Journal of vision.

[13]  I. Ohzawa,et al.  Receptive Field Properties of Neurons in the Early Visual Cortex Revealed by Local Spectral Reverse Correlation , 2006, The Journal of Neuroscience.

[14]  P. Lennie,et al.  Profound Contrast Adaptation Early in the Visual Pathway , 2004, Neuron.

[15]  M. Carandini.  Amplification of Trial-to-Trial Response Variability by Neurons in Visual Cortex , 2004, PLoS biology.

[16]  J. Movshon,et al.  Selectivity and spatial distribution of signals from the receptive field surround in macaque V1 neurons. , 2002, Journal of neurophysiology.

[17]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[18]  Wilson S. Geisler,et al.  Optimal defocus estimates from individual images for autofocusing a digital camera , 2012, Electronic Imaging.

[19]  W. Richards,et al.  Perception as Bayesian Inference , 2008 .

[20]  O. Schwartz,et al.  Flexible Gating of Contextual Influences in Natural Vision , 2015, Nature Neuroscience.

[21]  B. Knight,et al.  Contrast gain control in the primate retina: P cells are not X-like, some M cells are , 1992, Visual Neuroscience.

[22]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[23]  L. Croner,et al.  Receptive fields of P and M ganglion cells across the primate retina , 1995, Vision Research.

[24]  M. Cohen,et al.  Relating normalization to neuronal populations across cortical areas. , 2016, Journal of neurophysiology.

[25]  A. Parker,et al.  Spatial properties of neurons in the monkey striate cortex , 1987, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[26]  Anqi Wu,et al.  Convolutional spike-triggered covariance analysis for neural subunit models , 2015, NIPS.

[27]  D G Pelli,et al.  Uncertainty explains many aspects of visual contrast detection and discrimination. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[28]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[29]  Eero P. Simoncelli,et al.  Slow gain fluctuations limit benefits of temporal integration in visual cortex , 2018, Journal of vision.

[30]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[31]  Dwight A Burkhardt,et al.  Natural images and contrast encoding in bipolar cells in the retina of the land- and aquatic-phase tiger salamander , 2006, Visual Neuroscience.

[32]  F. e..  Calcul des Probabilités , 1889, Nature.

[33]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[34]  József Fiser,et al.  Coding of Natural Scenes in Primary Visual Cortex , 2003, Neuron.

[35]  Johannes Burge,et al.  Optimal speed estimation in natural image movies predicts human performance. , 2014, Journal of vision.

[36]  Fred Rieke,et al.  Review the Challenges Natural Images Pose for Visual Adaptation , 2022 .

[37]  Yves Frégnac,et al.  Animation of natural scene by virtual eye-movements evokes high precision and low noise in V1 neurons , 2013, Front. Neural Circuits.

[38]  Inés Samengo,et al.  Spike-triggered covariance: geometric proof, symmetry properties, and extension beyond Gaussian stimuli , 2012, Journal of Computational Neuroscience.

[39]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[40]  Ben Willmore,et al.  The Receptive-Field Organization of Simple Cells in Primary Visual Cortex of Ferrets under Natural Scene Stimulation , 2003, The Journal of Neuroscience.

[41]  Eero P. Simoncelli,et al.  Nonlinear image representation using divisive normalization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Garrett B Stanley,et al.  The episodic nature of spike trains in the early visual pathway. , 2010, Journal of neurophysiology.

[43]  M. A. Repucci,et al.  Spatial Structure and Symmetry of Simple-Cell Receptive Fields in Macaque Primary Visual Cortex , 2002 .

[44]  Eero P. Simoncelli,et al.  A Convolutional Subunit Model for Neuronal Responses in Macaque V1 , 2015, The Journal of Neuroscience.

[45]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[46]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[47]  Robert A. Frazor,et al.  Independence of luminance and contrast in natural scenes and in the early visual system , 2005, Nature Neuroscience.

[48]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[49]  E. Rossi,et al.  The relationship between visual resolution and cone spacing in the human fovea , 2009, Nature Neuroscience.

[50]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[51]  D. Pollen,et al.  Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey. , 1985, The Journal of physiology.

[52]  E. Chichilnisky,et al.  Adaptation to Temporal Contrast in Primate and Salamander Retina , 2001, The Journal of Neuroscience.

[53]  Eero P. Simoncelli,et al.  Testing pseudo-linear models of responses to natural scenes in primate retina , 2016, bioRxiv.

[54]  R. Shapley,et al.  The effect of contrast on the transfer properties of cat retinal ganglion cells. , 1978, The Journal of physiology.

[55]  L. Abbott,et al.  Responses of neurons in primary and inferior temporal visual cortices to natural scenes , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[56]  D. Tolhurst,et al.  Coding of the contrasts in natural images by populations of neurons in primary visual cortex (V1) , 2003, Vision Research.

[57]  Nicholas J. Priebe,et al.  Inhibition, Spike Threshold, and Stimulus Selectivity in Primary Visual Cortex , 2008, Neuron.

[58]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[59]  Peng Ding,et al.  On the Gaussian Mixture Representation of the Laplace Distribution , 2018 .

[60]  Johannes Burge,et al.  Optimal defocus estimation in individual natural images , 2011, Proceedings of the National Academy of Sciences.

[61]  Jiri Najemnik,et al.  Optimal stimulus encoders for natural tasks. , 2009, Journal of vision.

[62]  W. Geisler,et al.  Constrained sampling experiments reveal principles of detection in natural scenes , 2017, Proceedings of the National Academy of Sciences.

[63]  W. P. Tanner.  PHYSIOLOGICAL IMPLICATIONS OF PSYCHOPHYSICAL DATA * , 1961, Annals of the New York Academy of Sciences.

[64]  Michael S. Lewicki,et al.  Efficient coding of natural sounds , 2002, Nature Neuroscience.

[65]  Johannes Burge,et al.  Accuracy Maximization Analysis for Sensory-Perceptual Tasks: Computational Improvements, Filter Robustness, and Coding Advantages for Scaled Additive Noise , 2017, PLoS Comput. Biol..

[66]  R. L. Valois,et al.  The orientation and direction selectivity of cells in macaque visual cortex , 1982, Vision Research.

[67]  Brian C. McCann,et al.  Estimating 3D tilt from local image cues in natural scenes , 2016, Journal of vision.

[68]  Curtis L Baker,et al.  Natural versus Synthetic Stimuli for Estimating Receptive Field Models: A Comparison of Predictive Robustness , 2012, The Journal of Neuroscience.

[69]  H. Poincaré.  Calcul des Probabilités , 1912 .

[70]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[71]  M. Carandini,et al.  Normalization as a canonical neural computation , 2013, Nature Reviews Neuroscience.

[72]  D. G. Albrecht,et al.  Motion selectivity and the contrast-response function of simple cells in the visual cortex , 1991, Visual Neuroscience.

[73]  Yuzhi Chen,et al.  Sensory stimulation shifts visual cortex from synchronous to asynchronous states , 2014, Nature.

[74]  Feng Qi Han,et al.  Cortical Sensitivity to Visual Features in Natural Scenes , 2005, PLoS biology.

[75]  Martin Rehn,et al.  A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields , 2007, Journal of Computational Neuroscience.

[76]  A. Ahumada,et al.  Stimulus Features in Signal Detection , 1971 .

[77]  Eero P. Simoncelli,et al.  Spatiotemporal Elements of Macaque V1 Receptive Fields , 2005, Neuron.

[78]  G. F. Cooper,et al.  The angular selectivity of visual cortical cells to moving gratings , 1968, The Journal of physiology.

[79]  Eero P. Simoncelli,et al.  Modeling Multiscale Subbands of Photographic Images with Fields of Gaussian Scale Mixtures , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[80]  Mijung Park,et al.  Receptive Field Inference with Localized Priors , 2011, PLoS Comput. Biol..

[81]  Eero P. Simoncelli,et al.  Nonlinear Extraction of Independent Components of Natural Images Using Radial Gaussianization , 2009, Neural Computation.

[82]  Madineh Sedigh-Sarvestani,et al.  Inhibition in Simple Cell Receptive Fields Is Broad and OFF-Subregion Biased , 2017, The Journal of Neuroscience.

[83]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[84]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[85]  D. G. Albrecht,et al.  Visual cortex neurons in monkeys and cats: Detection, discrimination, and identification , 1997, Visual Neuroscience.

[86]  J. Victor,et al.  Population encoding of spatial frequency, orientation, and color in macaque V1. , 1994, Journal of neurophysiology.

[87]  Nicholas J Priebe,et al.  The accuracy of membrane potential reconstruction based on spiking receptive fields. , 2012, Journal of neurophysiology.

[88]  D J Field,et al.  Local Contrast in Natural Images: Normalisation and Coding Efficiency , 2000, Perception.

[89]  M. Carandini,et al.  Functional Mechanisms Shaping Lateral Geniculate Responses to Artificial and Natural Stimuli , 2008, Neuron.

[90]  Yuwei Cui,et al.  Inferring Nonlinear Neuronal Computation Based on Physiologically Plausible Inputs , 2013, PLoS Comput. Biol..

[91]  W. Geisler,et al.  Optimal disparity estimation in natural stereo images. , 2014, Journal of vision.