Spatially Pooled Contrast Responses Predict Neural and Perceptual Similarity of Naturalistic Image Categories

The visual world is complex and continuously changing. Yet, our brain transforms patterns of light falling on our retina into a coherent percept within a few hundred milliseconds. Possibly, low-level neural responses already carry substantial information to facilitate rapid characterization of the visual input. Here, we computationally estimated low-level contrast responses to computer-generated naturalistic images, and tested whether spatial pooling of these responses could predict image similarity at the neural and behavioral level. Using EEG, we show that statistics derived from pooled responses explain a large amount of variance between single-image evoked potentials (ERPs) in individual subjects. Dissimilarity analysis on multi-electrode ERPs demonstrated that large differences between images in pooled response statistics are predictive of more dissimilar patterns of evoked activity, whereas images with little difference in statistics give rise to highly similar evoked activity patterns. In a separate behavioral experiment, images with large differences in statistics were judged as different categories, whereas images with little differences were confused. These findings suggest that statistics derived from low-level contrast responses can be extracted in early visual processing and can be relevant for rapid judgment of visual similarity. We compared our results with two other, well- known contrast statistics: Fourier power spectra and higher-order properties of contrast distributions (skewness and kurtosis). Interestingly, whereas these statistics allow for accurate image categorization, they do not predict ERP response patterns or behavioral categorization confusions. These converging computational, neural and behavioral results suggest that statistics of pooled contrast responses contain information that corresponds with perceived visual similarity in a rapid, low-level categorization task.

[1]  N Mantel,et al.  A technique of nonparametric multivariate analysis. , 1970, Biometrics.

[2]  P. Sajda,et al.  Temporal characterization of the neural correlates of perceptual decision making in the human brain. , 2006, Cerebral cortex.

[3]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[4]  F. Perrin,et al.  Spherical splines for scalp potential and current density mapping. , 1989, Electroencephalography and clinical neurophysiology.

[5]  D. Field,et al.  Sensitivity to contrast histogram differences in synthetic wavelet-textures , 2001, Vision Research.

[6]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[7]  L. Chalupa,et al.  The visual neurosciences , 2004 .

[8]  R. Ratcliff,et al.  Neural Representation of Task Difficulty and Decision Making during Perceptual Categorization: A Timing Diagram , 2006, The Journal of Neuroscience.

[9]  Lester C. Loschky,et al.  Localized information is necessary for scene categorization, including the Natural/Man-made distinction. , 2008, Journal of vision.

[10]  Jesse S. Husk,et al.  Spatial scaling factors explain eccentricity effects on face ERPs. , 2005, Journal of vision.

[11]  A. Harris,et al.  Temporally distinct neural coding of perceptual similarity and prototype bias. , 2010, Journal of vision.

[12]  R. Vogels,et al.  The representation of perceived shape similarity and its role for category learning in monkeys: A modeling study , 2008, Vision Research.

[13]  G. Rousselet,et al.  How do amplitude spectra influence rapid animal detection? , 2009, Vision Research.

[14]  Jesse S. Husk,et al.  Time course and robustness of ERP object and face differences. , 2008, Journal of vision.

[15]  Guillaume A. Rousselet,et al.  Reliability of ERP and single-trial analyses , 2011, NeuroImage.

[16]  Aude Oliva,et al.  Estimating perception of scene layout properties from global image features. , 2011, Journal of vision.

[17]  Philippe G. Schyns,et al.  Dynamics of Trimming the Content of Face Representations for Categorization in the Brain , 2009, PLoS Comput. Biol..

[18]  Arnold W. M. Smeulders,et al.  A Biologically Plausible Model for Rapid Natural Scene Identification , 2009, NIPS.

[19]  Jitendra Malik,et al.  When is scene identification just texture recognition? , 2004, Vision Research.

[20]  Michael S. Lewicki,et al.  Relations between the statistical regularities of natural images and the response properties of the early visual system , 2005 .

[21]  Stefan Treue,et al.  Adaptation to statistical properties of visual scenes biases rapid categorization , 2007 .

[22]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[23]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[24]  Bruce C. Hansen,et al.  From spatial frequency contrast to edge preponderance: the differential modulation of early visual evoked potentials by natural scene stimuli , 2011, Visual Neuroscience.

[25]  G. Aguirre,et al.  Different spatial scales of shape similarity representation in lateral and ventral LOC. , 2009, Cerebral cortex.

[26]  Andriana Olmos,et al.  A biologically inspired algorithm for the recovery of shading and reflectance images , 2004 .

[27]  E Donchin,et al.  A new method for off-line removal of ocular artifact. , 1983, Electroencephalography and clinical neurophysiology.

[28]  Leslie G. Ungerleider,et al.  Texture segregation in the human visual cortex: A functional MRI study. , 2000, Journal of neurophysiology.

[29]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[30]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[31]  Guillaume A. Rousselet,et al.  Quantifying the Time Course of Visual Object Processing Using ERPs: It's Time to Up the Game , 2011, Front. Psychology.

[32]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[33]  Jasna Martinovic,et al.  Event-related potentials reveal an early advantage for luminance contours in the processing of objects. , 2011, Journal of vision.

[34]  D J Field,et al.  Local Contrast in Natural Images: Normalisation and Coding Efficiency , 2000, Perception.

[35]  Eero P. Simoncelli,et al.  Metamers of the ventral stream , 2011, Nature Neuroscience.

[36]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[37]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[38]  L. Croner,et al.  Receptive fields of P and M ganglion cells across the primate retina , 1995, Vision Research.

[39]  Simon J Thorpe,et al.  The Speed of Categorization in the Human Visual System , 2009, Neuron.

[40]  D. Tolhurst,et al.  Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes , 2000, Vision Research.

[41]  D. Ruderman.  The statistics of natural images , 1994 .

[42]  R Goebel,et al.  Inverse mapping the neuronal substrates of face categorizations. , 2009, Cerebral cortex.

[43]  W. Geisler.  Visual perception and the statistical properties of natural scenes. , 2008, Annual review of psychology.

[44]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[45]  Michael S. Lewicki,et al.  Emergence of complex cell properties by learning to generalize in natural scenes , 2009, Nature.

[46]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[47]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[48]  H. Wool.  THE RELATION BETWEEN MEASURES OF CORRELATION IN THE UNIVERSE OF SAMPLE PERMUTATIONS , 1944 .

[49]  I. Biederman.  Perceiving Real-World Scenes , 1972, Science.

[50]  M. Potter.  Meaning in visual search. , 1975, Science.

[51]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[52]  Bruno A Olshausen,et al.  Timecourse of neural signatures of object recognition. , 2003, Journal of vision.

[53]  M. Carandini,et al.  The Suppressive Field of Neurons in Lateral Geniculate Nucleus , 2005, The Journal of Neuroscience.

[54]  Robert A. Frazor,et al.  Local luminance and contrast in natural images , 2006, Vision Research.

[55]  Abel G. Oliva,et al.  Gist of a scene , 2005 .

[56]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[57]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[58]  Arnold W. M. Smeulders,et al.  c ○ 2005 Springer Science + Business Media, Inc. Manufactured in The Netherlands. A Six-Stimulus Theory for Stochastic Texture , 2002 .

[59]  Victor A. F. Lamme,et al.  Low-level contrast statistics are diagnostic of invariance of natural textures , 2012, Front. Comput. Neurosci..

[60]  Dwight J. Kravitz,et al.  Real-World Scene Representations in High-Level Visual Cortex: It's the Spaces More Than the Places , 2011, The Journal of Neuroscience.

[61]  R. Millane,et al.  Effects of occlusion, edges, and scaling on the power spectra of natural images. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[62]  A. Oliva,et al.  Coarse Blobs or Fine Edges? Evidence That Information Diagnosticity Changes the Perception of Complex Visual Stimuli , 1997, Cognitive Psychology.

[63]  D. Braun,et al.  Phase noise and the classification of natural images , 2006, Vision Research.

[64]  P. Schyns,et al.  Cracking the Code of Oscillatory Activity , 2011, PLoS biology.

[65]  S. Thorpe,et al.  The time course of visual processing: Backward masking and natural scene categorisation , 2005, Vision Research.

[66]  Paul H. Garthwaite,et al.  CONFIDENCE INTERVALS FROM RANDOMIZATION TESTS , 1996 .

[67]  Michelle R. Greene,et al.  PSYCHOLOGICAL SCIENCE Research Article The Briefest of Glances The Time Course of Natural Scene Understanding , 2022 .

[68]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[69]  Steven W. Zucker,et al.  Local Scale Control for Edge Detection and Blur Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  R. VanRullen,et al.  Faces in the cloud: Fourier power spectrum biases ultrarapid face detection. , 2008, Journal of vision.

[71]  S. Thorpe,et al.  The Time Course of Visual Processing: From Early Perception to Decision-Making , 2001, Journal of Cognitive Neuroscience.

[72]  R. Shepard.  Attention and the metric structure of the stimulus space. , 1964 .

[73]  N. Graham.  Does the brain perform a Fourier analysis of the visual scene? , 1979, Trends in Neurosciences.

[74]  A. Smeulders,et al.  A Biologically Plausible Model for Rapid Natural Image Identi cation , 2009 .

[75]  Michael S. Landy,et al.  Visual perception of texture , 2002 .

[76]  Johannes J. Fahrenfort,et al.  Feedforward and Recurrent Processing in Scene Segmentation: Electroencephalography and Functional Magnetic Resonance Imaging , 2008, Journal of Cognitive Neuroscience.

[77]  Guillaume A. Rousselet,et al.  Parallel processing in high-level categorization of natural images , 2002, Nature Neuroscience.

[78]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[79]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[80]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[81]  Li Fei-Fei,et al.  Neural mechanisms of rapid natural scene categorization in human visual cortex , 2009, Nature.

[82]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[83]  P. Perona,et al.  Rapid natural scene categorization in the near absence of attention , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[84]  G. Rousselet,et al.  Parametric study of EEG sensitivity to phase noise during face processing , 2008, BMC Neuroscience.

[85]  Arnold W. M. Smeulders,et al.  Brain responses strongly correlate with Weibull image statistics when processing natural images. , 2009, Journal of vision.

[86]  Simon J. Thorpe,et al.  Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited , 2006, Vision Research.

[87]  Guillaume A. Rousselet,et al.  Rapid visual categorization of natural scene contexts with equalized amplitude spectrum and increasing phase noise. , 2009, Journal of vision.