Common spatiotemporal processing of visual features shapes object representation

Biological vision relies on representations of the physical world at different levels of complexity. Relevant features span from simple low-level properties, as contrast and spatial frequencies, to object-based attributes, as shape and category. However, how these features are integrated into coherent percepts is still debated. Moreover, these dimensions often share common biases: for instance, stimuli from the same category (e.g., tools) may have similar shapes. Here, using magnetoencephalography, we revealed the temporal dynamics of feature processing in human subjects attending to pictures of items pertaining to different semantic categories. By employing Relative Weights Analysis, we mitigated collinearity between model-based descriptions of stimuli and showed that low-level properties (contrast and spatial frequencies), shape (medial-axis) and category are represented within the same spatial locations early in time: 100-150ms after stimulus onset. This fast and overlapping processing may result from independent parallel computations, with categorical representation emerging later than the onset of low-level feature processing, yet before shape coding. Categorical information is represented both before and after shape also suggesting a role for this feature in the refinement of categorical matching.

[1]  Riitta Salmelin,et al.  Tracking neural coding of perceptual and semantic features of concrete nouns , 2012, NeuroImage.

[2]  J. W. Johnson A Heuristic Method for Estimating the Relative Weight of Predictor Variables in Multiple Regression , 2000, Multivariate behavioral research.

[3]  R. Passingham,et al.  Objects automatically potentiate action: an fMRI study of implicit processing , 2003, The European journal of neuroscience.

[4]  Stephen M. Smith,et al.  Threshold-free cluster enhancement: Addressing problems of smoothing, threshold dependence and localisation in cluster inference , 2009, NeuroImage.

[5]  Tom Hartley,et al.  Low-Level Image Properties of Visual Objects Predict Patterns of Neural Response across Category-Selective Regions of the Ventral Visual Pathway , 2014, The Journal of Neuroscience.

[6]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[7]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[8]  I. Biederman Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[9]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[10]  Bruno A Olshausen,et al.  Timecourse of neural signatures of object recognition. , 2003, Journal of vision.

[11]  Johan Wagemans,et al.  A conceptual framework of computations in mid-level vision , 2014, Front. Comput. Neurosci..

[12]  Peter Neri,et al.  Global Properties of Natural Scenes Shape Local Properties of Human Edge Detectors , 2011, Front. Psychology.

[13]  H. P. Op de Beeck,et al.  Dissociations and Associations between Shape and Category Representations in the Two Visual Pathways , 2015, The Journal of Neuroscience.

[14]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[15]  Zijiang J. He,et al.  Vertical and horizontal references determined by linear perspective and optic flow information , 2010 .

[16]  L. Tyler,et al.  Decoding the Cortical Dynamics of Sound-Meaning Mapping , 2017, The Journal of Neuroscience.

[17]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[18]  H. Neumann,et al.  The Role of Attention in Figure-Ground Segregation in Areas V1 and V4 of the Visual Cortex , 2012, Neuron.

[19]  James V. Haxby,et al.  CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave , 2016, bioRxiv.

[20]  G. Rizzolatti,et al.  Evidence for visuomotor priming effect , 1996, Neuroreport.

[21]  P. Neri Semantic Control of Feature Extraction from Natural Scenes , 2014, The Journal of Neuroscience.

[22]  John A. Pyles,et al.  Comparing visual representations across human fMRI and computational vision. , 2013, Journal of vision.

[23]  F. Qiu,et al.  Figure-ground mechanisms provide structure for selective attention , 2007, Nature Neuroscience.

[24]  L. Tyler,et al.  Predicting the Time Course of Individual Objects with MEG , 2014, Cerebral cortex.

[25]  V. Lamme,et al.  The distinct modes of vision offered by feedforward and recurrent processing , 2000, Trends in Neurosciences.

[26]  Thomas E. Nichols,et al.  luster-based computational methods for mass univariate analyses f event-related brain potentials / fields : A simulation study , 2022 .

[27]  Kim F. Nimon,et al.  Understanding the Results of Multiple Linear Regression , 2013 .

[28]  Li Fei-Fei,et al.  Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior , 2018, eLife.

[29]  Philip N. Klein,et al.  Recognition of shapes by editing their shock graphs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Pieter R. Roelfsema,et al.  Texture Segregation Causes Early Figure Enhancement and Later Ground Suppression in Areas V1 and V4 of Visual Cortex , 2016, Cerebral cortex.

[31]  H. Blum Biological shape and visual science. I. , 1973, Journal of theoretical biology.

[32]  Doris Y. Tsao,et al.  Consistency of Border-Ownership Cells across Artificial Stimuli, Natural Stimuli, and Stimuli with Ambiguous Contours , 2016, The Journal of Neuroscience.

[33]  Victor A. F. Lamme The neurophysiology of figure-ground segregation in primary visual cortex , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[34]  Radoslaw Martin Cichy,et al.  The representational dynamics of task and object processing in humans , 2018, eLife.

[35]  Emiliano Ricciardi,et al.  Modality-independent encoding of individual concepts in the left parietal cortex , 2017, Neuropsychologia.

[36]  D. Field,et al.  Natural image statistics and efficient coding. , 1996, Network.

[37]  George L. Malcolm,et al.  Making Sense of Real-World Scenes , 2016, Trends in Cognitive Sciences.

[38]  O. Schwartz,et al.  Visual attention and flexible normalization pools. , 2013, Journal of vision.

[39]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[40]  Peter Neri,et al.  Object segmentation controls image reconstruction from natural scenes , 2017, PLoS biology.

[41]  Eric T. Carlson,et al.  Medial Axis Shape Coding in Macaque Inferotemporal Cortex , 2012, Neuron.

[42]  Emiliano Ricciardi,et al.  Foreground-Background Segmentation Revealed during Natural Image Viewing , 2017, eNeuro.

[43]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[44]  Marius V Peelen,et al.  Shape-independent object category responses revealed by MEG and fMRI decoding. , 2016, Journal of neurophysiology.

[45]  Irving Biederman,et al.  Cortical representation of medial axis structure. , 2013, Cerebral cortex.

[46]  M. Bar A Cortical Mechanism for Triggering Top-Down Facilitation in Visual Object Recognition , 2003, Journal of Cognitive Neuroscience.

[47]  Kechen Zhang,et al.  A Sparse Object Coding Scheme in Area V4 , 2011, Current Biology.

[48]  H. Blum Biological shape and visual science (part I) , 1973 .

[49]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[50]  Kim Nimon,et al.  Using commonality analysis in multiple regressions: a tool to decompose regression effects in the face of multicollinearity , 2014 .

[51]  Daria Proklova,et al.  Disentangling Representations of Object Shape and Object Category in Human Visual Cortex: The Animate–Inanimate Distinction , 2016, Journal of Cognitive Neuroscience.

[52]  Radoslaw Martin Cichy,et al.  Multivariate pattern analysis for MEG: A comparison of dissimilarity measures , 2018, NeuroImage.

[53]  Jack L. Gallant,et al.  Fourier power, subjective distance, and object categories all provide plausible models of BOLD responses in scene-selective visual areas , 2015, Front. Comput. Neurosci..

[54]  Emiliano Ricciardi,et al.  How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge , 2016, NeuroImage.

[55]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[56]  Dwight J. Kravitz,et al.  Task context impacts visual object processing differentially across the cortex , 2014, Proceedings of the National Academy of Sciences.

[57]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[58]  Jonathan R. Williford,et al.  Figure-ground organization in visual cortex for natural scenes , 2016 .

[59]  Joachim Gross,et al.  Good practice for conducting and reporting MEG research , 2013, NeuroImage.