Resolving human object recognition in space and time

A comprehensive picture of object processing in the human brain requires combining both spatial and temporal information about brain activity. Here we acquired human magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) responses to 92 object images. Multivariate pattern classification applied to MEG revealed the time course of object processing: whereas individual images were discriminated by visual representations early, ordinate and superordinate category levels emerged relatively late. Using representational similarity analysis, we combined human fMRI and MEG to show content-specific correspondence between early MEG responses and primary visual cortex (V1), and later MEG responses and inferior temporal (IT) cortex. We identified transient and persistent neural activities during object processing with sources in V1 and IT. Finally, we correlated human MEG signals to single-unit responses in monkey IT. Together, our findings provide an integrated space- and time-resolved view of human object categorization during the first few hundred milliseconds of vision.

[1]  G. Schneider.  Two visual systems. , 1969, Science.

[2]  R N Shepard,et al.  Multidimensional Scaling, Tree-Fitting, and Clustering , 1980, Science.

[3]  R. Mansfield,et al.  Analysis of visual behavior , 1982 .

[4]  P. Grobstein.  Analysis of Visual Behavior, David J. Ingle, Melvyn A. Goodale, Richard J.W. Mansfield (Eds.). MIT press, Cambridge, MA and London (1982), 834 , 1983 .

[5]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[6]  T. Allison,et al.  Face recognition in human extrastriate cortex. , 1994, Journal of neurophysiology.

[7]  M. Goodale,et al.  The visual brain in action , 1995 .

[8]  T. Allison,et al.  Electrophysiological Studies of Face Perception in Humans , 1996, Journal of Cognitive Neuroscience.

[9]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[10]  D. Jeffreys.  Evoked Potential Studies of Face and Object Processing , 1996 .

[11]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[12]  S Edelman,et al.  Representation is representation of similarities , 1996, Behavioral and Brain Sciences.

[13]  A. Leventhal,et al.  Signal timing across the macaque visual system. , 1998, Journal of neurophysiology.

[14]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[15]  T. Allison,et al.  Electrophysiological studies of human face perception. II: Response properties of face-specific potentials generated in occipitotemporal cortex. , 1999, Cerebral cortex.

[16]  E. Halgren,et al.  Dynamic Statistical Parametric Mapping Combining fMRI and MEG for High-Resolution Imaging of Cortical Activity , 2000, Neuron.

[17]  E. Halgren,et al.  Cognitive response profile of the human fusiform face area as determined by MEG. , 2000, Cerebral cortex.

[18]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[19]  N. Kanwisher,et al.  The Human Body , 2001 .

[20]  Richard M. Leahy,et al.  Electromagnetic brain mapping , 2001, IEEE Signal Process. Mag..

[21]  S. Thorpe,et al.  The Time Course of Visual Processing: From Early Perception to Decision-Making , 2001, Journal of Cognitive Neuroscience.

[22]  N. Kanwisher,et al.  Stages of processing in face perception: an MEG study , 2002, Nature Neuroscience.

[23]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[24]  M. Tarr,et al.  Visual Object Recognition , 1996, ISTCS.

[25]  Paul J. Laurienti,et al.  An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets , 2003, NeuroImage.

[26]  J. Stekelenburg,et al.  The neural correlates of perceiving human bodies: an ERP study on the body-inversion effect , 2004, Neuroreport.

[27]  K. Grill-Spector,et al.  The human visual cortex. , 2004, Annual review of neuroscience.

[28]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[29]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[30]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[31]  Scott L. Brincat,et al.  Dynamic Shape Synthesis in Posterior Inferotemporal Cortex , 2006, Neuron.

[32]  Paul E. Downing,et al.  An event-related potential component sensitive to images of the human body , 2006, NeuroImage.

[33]  A. Engel,et al.  Single-trial EEG–fMRI reveals the dynamics of cognitive function , 2006, Trends in Cognitive Sciences.

[34]  Keiji Tanaka,et al.  Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. , 2007, Journal of neurophysiology.

[35]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[36]  C. Koch,et al.  Latency and Selectivity of Single Neurons Indicate Hierarchical Processing in the Human Medial Temporal Lobe , 2008, The Journal of Neuroscience.

[37]  Y. Nakayama,et al.  The history of JOV of 10 years , 2008, J. Vis..

[38]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[39]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[40]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[41]  G. Kreiman,et al.  Timing, Timing, Timing: Fast Decoding of Object Information from Intracranial Field Potentials in Human Visual Cortex , 2009, Neuron.

[42]  F. Tong,et al.  Decoding reveals the contents of visual working memory in early visual areas , 2009, Nature.

[43]  Masaaki Kawahashi,et al.  Renovation of Journal of Visualization , 2010, J. Vis..

[44]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[45]  G. Yovel,et al.  Event‐related potential and functional MRI measures of face‐selectivity are highly correlated: A simultaneous ERP‐fMRI investigation , 2010, Human brain mapping.

[46]  K. Kawano,et al.  Role of Temporal Processing Stages by Inferior Temporal Neurons in Facial Recognition , 2011, Front. Psychology.

[47]  T. Carlson,et al.  High temporal resolution decoding of object position and category. , 2011, Journal of vision.

[48]  C. Connor,et al.  Neural representations for object perception: structure, category, and adaptive coding. , 2011, Annual review of neuroscience.

[49]  Michael L. Mack,et al.  The Timing of Visual Object Categorization , 2011, Front. Psychology.

[50]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[51]  J. Haynes.  Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[52]  Riitta Salmelin,et al.  Magnetoencephalography: From SQUIDs to neuroscience Neuroimage 20th Anniversary Special Edition , 2012, NeuroImage.

[53]  James J. DiCarlo,et al.  How Does the Brain Solve Visual Object Recognition? , 2012, Neuron.

[54]  Omar H. Butt,et al.  The Retinotopic Organization of Striate Cortex Is Well Predicted by Surface Topology , 2012, Current Biology.

[55]  Michael S. Pratte,et al.  Decoding patterns of human brain activity. , 2012, Annual review of psychology.

[56]  Michael C. Hout,et al.  Multidimensional Scaling , 2003, Encyclopedic Dictionary of Archaeology.

[57]  Dwight J. Kravitz,et al.  The ventral visual pathway: an expanded neural framework for the processing of object quality , 2013, Trends in Cognitive Sciences.

[58]  David A. Tovar,et al.  Representational dynamics of object vision: the first 1000 ms. , 2013, Journal of vision.

[59]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .