Model neuron response statistics to natural images

To model the responses of neurons in the early visual system, at least three basic components are required: a receptive field, a normalization term, and a specification of encoding noise. Here, we examine how the receptive field, the normalization factor, and the encoding noise impact the model neuron responses to natural images and the signal-to-noise ratio for natural image discrimination. We show that when these components are modeled appropriately, the model neuron responses to natural stimuli are Gaussian distributed, scale-invariant, and very nearly maximize the signal-to-noise ratio for stimulus discrimination. We discuss the statistical models of natural stimuli that can account for these response statistics, and we show how some commonly used modeling practices may distort these results. Finally, we show that normalization can equalize important properties of neural response across different stimulus types. Specifically, narrowband (stimulus- and feature-specific) normalization causes model neurons to yield Gaussian-distributed responses to natural stimuli, 1/f noise stimuli, and white noise stimuli. The current work makes recommendations for best practices and it lays a foundation, grounded in the response statistics to natural stimuli, upon which principled models of more complex visual tasks can be built.

[1]  Fred Rieke,et al.  Review the Challenges Natural Images Pose for Visual Adaptation , 2022 .

[2]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[3]  R. L. Valois,et al.  The orientation and direction selectivity of cells in macaque visual cortex , 1982, Vision Research.

[4]  M. Carandini.  Amplification of Trial-to-Trial Response Variability by Neurons in Visual Cortex , 2004, PLoS biology.

[5]  Adam Binch,et al.  Perception as Bayesian Inference , 2014 .

[6]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[7]  F. e..  Calcul des Probabilités , 1889, Nature.

[8]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[9]  L. Croner,et al.  Receptive fields of P and M ganglion cells across the primate retina , 1995, Vision Research.

[10]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[11]  Charless C. Fowlkes,et al.  Natural-Scene Statistics Predict How the Figure–Ground Cue of Convexity Affects Human Depth Perception , 2010, The Journal of Neuroscience.

[12]  Timothy A. Machado,et al.  Functional connectivity in the retina at the resolution of photoreceptors , 2010, Nature.

[13]  Eero P. Simoncelli,et al.  A Convolutional Subunit Model for Neuronal Responses in Macaque V1 , 2015, The Journal of Neuroscience.

[14]  P. Lennie,et al.  Profound Contrast Adaptation Early in the Visual Pathway , 2004, Neuron.

[15]  Simon Barthelmé,et al.  Improved classification images with sparse priors in a smooth basis. , 2009, Journal of vision.

[16]  B. Knight,et al.  Contrast gain control in the primate retina: P cells are not X-like, some M cells are , 1992, Visual Neuroscience.

[17]  Johannes Burge,et al.  Optimal defocus estimation in individual natural images , 2011, Proceedings of the National Academy of Sciences.

[18]  S. Morad,et al.  Ceramide-orchestrated signalling in cancer cells , 2012, Nature Reviews Cancer.

[19]  Robert A. Frazor,et al.  Independence of luminance and contrast in natural scenes and in the early visual system , 2005, Nature Neuroscience.

[20]  Jiri Najemnik,et al.  Optimal stimulus encoders for natural tasks. , 2009, Journal of vision.

[21]  W. Geisler,et al.  Constrained sampling experiments reveal principles of detection in natural scenes , 2017, Proceedings of the National Academy of Sciences.

[22]  Su Keun Jeong,et al.  The impact of top-down spatial attention on laterality and hemispheric asymmetry in the human parietal cortex , 2016, Journal of vision.

[23]  Katja Doerschner,et al.  Seeing through transparent layers. , 2017, Journal of vision.

[24]  Edward H. Adelson,et al.  Motion illusions as optimal percepts , 2002, Nature Neuroscience.

[25]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[26]  Peng Ding,et al.  On the Gaussian Mixture Representation of the Laplace Distribution , 2018 .

[27]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[28]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[29]  Eero P. Simoncelli,et al.  How MT cells analyze the motion of visual patterns , 2006, Nature Neuroscience.

[30]  Alan A. Stocker,et al.  Signal Integration in Human Visual Speed Perception , 2015, The Journal of Neuroscience.

[31]  D. G. Albrecht,et al.  Visual cortex neurons in monkeys and cats: Detection, discrimination, and identification , 1997, Visual Neuroscience.

[32]  J. Victor,et al.  Population encoding of spatial frequency, orientation, and color in macaque V1. , 1994, Journal of neurophysiology.

[33]  Johannes Burge,et al.  Defocus blur discrimination in natural images with natural optics. , 2015, Journal of vision.

[34]  Inés Samengo,et al.  Spike-triggered covariance: geometric proof, symmetry properties, and extension beyond Gaussian stimuli , 2012, Journal of Computational Neuroscience.

[35]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[36]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[37]  Brian C. McCann,et al.  Estimating 3D tilt from local image cues in natural scenes , 2016, Journal of vision.

[38]  Feng Qi Han,et al.  Cortical Sensitivity to Visual Features in Natural Scenes , 2005, PLoS biology.

[39]  E. Rossi,et al.  The relationship between visual resolution and cone spacing in the human fovea , 2009, Nature Neuroscience.

[40]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[41]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[42]  Emily A. Cooper,et al.  Blur and Disparity Are Complementary Cues to Depth , 2012, Current Biology.

[43]  D. G. Albrecht,et al.  Motion selectivity and the contrast-response function of simple cells in the visual cortex , 1991, Visual Neuroscience.

[44]  Curtis L Baker,et al.  Natural versus Synthetic Stimuli for Estimating Receptive Field Models: A Comparison of Predictive Robustness , 2012, The Journal of Neuroscience.

[45]  Wilson S. Geisler,et al.  Optimal defocus estimates from individual images for autofocusing a digital camera , 2012, Electronic Imaging.

[46]  D I Flitcroft,et al.  A neural and computational model for the chromatic control of accommodation , 1990, Visual Neuroscience.

[47]  Christopher W. Tyler,et al.  Binocular cross-correlation in time and space , 1978, Vision Research.

[48]  Nicholas J. Priebe,et al.  Inhibition, Spike Threshold, and Stimulus Selectivity in Primary Visual Cortex , 2008, Neuron.

[49]  Michael S. Lewicki,et al.  Efficient coding of natural sounds , 2002, Nature Neuroscience.

[50]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[51]  Johannes Burge,et al.  Accuracy Maximization Analysis for Sensory-Perceptual Tasks: Computational Improvements, Filter Robustness, and Coding Advantages for Scaled Additive Noise , 2017, PLoS Comput. Biol..

[52]  Wilson S. Geisler,et al.  Optimal speed estimation in natural image movies predicts human performance , 2015, Nature Communications.

[53]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[54]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[55]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[56]  M. Landy,et al.  Why Is Spatial Stereoresolution So Low? , 2004, The Journal of Neuroscience.

[57]  L. Cormack,et al.  Interocular correlation, luminance contrast and cyclopean processing , 1991, Vision Research.

[58]  J. Alonso,et al.  Adaptation to Stimulus Contrast and Correlations during Natural Visual Stimulation , 2007, Neuron.

[59]  Eero P. Simoncelli,et al.  Spike-triggered neural characterization. , 2006, Journal of vision.

[60]  W. Geisler,et al.  Optimal disparity estimation in natural stereo images. , 2014, Journal of vision.

[61]  I. Ohzawa,et al.  Receptive Field Properties of Neurons in the Early Visual Cortex Revealed by Local Spectral Reverse Correlation , 2006, The Journal of Neuroscience.

[62]  Pascal Mamassian,et al.  A Normalization Mechanism for Estimating Visual Motion across Speeds and Scales , 2017, Current Biology.

[63]  Martin J. Wainwright,et al.  Scale Mixtures of Gaussians and the Statistics of Natural Images , 1999, NIPS.

[64]  Ben Willmore,et al.  The Receptive-Field Organization of Simple Cells in Primary Visual Cortex of Ferrets under Natural Scene Stimulation , 2003, The Journal of Neuroscience.

[65]  A. Parker,et al.  Spatial properties of neurons in the monkey striate cortex , 1987, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[66]  Nicholas J. Priebe,et al.  The Neural Representation of Speed in Macaque Area MT/V5 , 2003, The Journal of Neuroscience.

[67]  D G Pelli,et al.  Uncertainty explains many aspects of visual contrast detection and discrimination. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[68]  Johannes Burge,et al.  Depth variation and stereo processing tasks in natural scenes , 2018, Journal of vision.

[69]  Jonathan W. Pillow,et al.  Spectral methods for neural characterization using generalized quadratic models , 2013, NIPS.

[70]  D. Tolhurst,et al.  Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes , 2000, Vision Research.

[71]  Nathalie Guyader,et al.  Blind search — successful saccades to the unknown target location up to 1000 ms after removal of visual search stimulus , 2010 .

[72]  R. Caldara,et al.  The Facespan-the perceptual span for face recognition. , 2017, Journal of vision.

[73]  D. Pollen,et al.  Spatial and temporal frequency selectivity of neurones in visual cortical areas V1 and V2 of the macaque monkey. , 1985, The Journal of physiology.

[74]  E. Chichilnisky,et al.  Adaptation to Temporal Contrast in Primate and Salamander Retina , 2001, The Journal of Neuroscience.

[75]  Eero P. Simoncelli,et al.  Testing pseudo-linear models of responses to natural scenes in primate retina , 2016, bioRxiv.

[76]  D. Tolhurst,et al.  Coding of the contrasts in natural images by populations of neurons in primary visual cortex (V1) , 2003, Vision Research.

[77]  Eero P. Simoncelli,et al.  Spatiotemporal Elements of Macaque V1 Receptive Fields , 2005, Neuron.

[78]  Nicholas J. Priebe,et al.  Estimating Target Speed from the Population Response in Visual Area MT , 2004, The Journal of Neuroscience.

[79]  G. F. Cooper,et al.  The angular selectivity of visual cortical cells to moving gratings , 1968, The Journal of physiology.

[80]  Eero P. Simoncelli,et al.  Modeling Multiscale Subbands of Photographic Images with Fields of Gaussian Scale Mixtures , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Mijung Park,et al.  Receptive Field Inference with Localized Priors , 2011, PLoS Comput. Biol..

[82]  Johannes Burge,et al.  Ordinal configural cues combine with metric disparity in depth perception. , 2005, Journal of vision.

[83]  Johannes Burge,et al.  Linking normative models of natural tasks to descriptive models of neural response , 2017, bioRxiv.

[84]  A. Logvinenko.  The geometric structure of color. , 2015, Journal of vision.

[85]  Eero P. Simoncelli,et al.  Nonlinear image representation using divisive normalization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[86]  Garrett B Stanley,et al.  The episodic nature of spike trains in the early visual pathway. , 2010, Journal of neurophysiology.

[87]  RussLL L. Ds Vnlos,et al.  SPATIAL FREQUENCY SELECTIVITY OF CELLS IN MACAQUE VISUAL CORTEX , 2022 .

[88]  Martin Rehn,et al.  A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields , 2007, Journal of Computational Neuroscience.

[89]  A. Ahumada,et al.  Stimulus Features in Signal Detection , 1971 .

[90]  Nicholas J Priebe,et al.  The accuracy of membrane potential reconstruction based on spiking receptive fields. , 2012, Journal of neurophysiology.

[91]  D J Field,et al.  Local Contrast in Natural Images: Normalisation and Coding Efficiency , 2000, Perception.

[92]  M. Carandini,et al.  Functional Mechanisms Shaping Lateral Geniculate Responses to Artificial and Natural Stimuli , 2008, Neuron.

[93]  Yuwei Cui,et al.  Inferring Nonlinear Neuronal Computation Based on Physiologically Plausible Inputs , 2013, PLoS Comput. Biol..

[94]  Yves Frégnac,et al.  Animation of natural scene by virtual eye-movements evokes high precision and low noise in V1 neurons , 2013, Front. Neural Circuits.

[95]  P B Kruger,et al.  Accommodation to Stationary and Moving Targets , 1997, Optometry and vision science : official publication of the American Academy of Optometry.

[96]  Eero P. Simoncelli,et al.  Nonlinear Extraction of Independent Components of Natural Images Using Radial Gaussianization , 2009, Neural Computation.

[97]  Madineh Sedigh-Sarvestani,et al.  Inhibition in Simple Cell Receptive Fields Is Broad and OFF-Subregion Biased , 2017, The Journal of Neuroscience.

[98]  J. C. Kotulak,et al.  A computational model of the error detector of human visual accommodation , 1986, Biological Cybernetics.

[99]  F. Attneave.  Some informational aspects of visual perception. , 1954, Psychological review.

[100]  Dwight A Burkhardt,et al.  Natural images and contrast encoding in bipolar cells in the retina of the land- and aquatic-phase tiger salamander , 2006, Visual Neuroscience.

[101]  József Fiser,et al.  Coding of Natural Scenes in Primary Visual Cortex , 2003, Neuron.

[102]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[103]  R. Shapley,et al.  The effect of contrast on the transfer properties of cat retinal ganglion cells. , 1978, The Journal of physiology.