Dynamic Construction of Reduced Representations in the Brain for Perceptual Decision Behavior

Current models propose that the brain uses a multi-layered architecture to reduce the high dimensional visual input to lower dimensional representations that support face, object and scene categorizations. However, understanding the brain mechanisms that support such information reduction for behavior remains challenging. We addressed the challenge using a novel information theoretic framework that quantifies the relationships between three key variables: single-trial information randomly sampled from an ambiguous scene, source-space MEG responses and perceptual decision behaviors. In each observer, behavioral analysis revealed the scene features that subtend their decisions. Independent source space analyses revealed the flow of these and other features in cortical activity. We show where (at the junction between occipital cortex and ventral regions), when (up until 170 ms post stimulus) and how (by separating task-relevant and irrelevant features) brain regions reduce the high-dimensional scene to construct task-relevant feature representations in the right fusiform gyrus that support decisions. Our results inform the occipito-temporal pathway mechanisms that reduce and select information to produce behavior.

[1]  Cheryl Olman,et al.  Classification objects, ideal observers & generative models , 2004, Cogn. Sci..

[2]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[3]  Dwight J. Kravitz,et al.  The ventral visual pathway: an expanded neural framework for the processing of object quality , 2013, Trends in Cognitive Sciences.

[4]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[5]  P. Schyns,et al.  Measuring Internal Representations from Behavioral and Brain Data , 2012, Current Biology.

[6]  W. Drongelen,et al.  Localization of brain electrical activity via linearly constrained minimum variance spatial filtering , 1997, IEEE Transactions on Biomedical Engineering.

[7]  Matthias Niemeier,et al.  A contralateral preference in the lateral occipital area: sensory and attentional mechanisms. , 2004, Cerebral cortex.

[8]  Guillaume A. Rousselet,et al.  Tracing the Flow of Perceptual Features in an Algorithmic Brain Network , 2015, Scientific Reports.

[9]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[10]  Lucy S. Petro,et al.  Dynamics of Visual Information Integration in the Brain for Categorizing Facial Expressions , 2007, Current Biology.

[11]  Dirk B. Walther,et al.  Natural Scene Categories Revealed in Distributed Patterns of Activity in the Human Brain , 2009, The Journal of Neuroscience.

[12]  S. Thorpe,et al.  The Time Course of Visual Processing: From Early Perception to Decision-Making , 2001, Journal of Cognitive Neuroscience.

[13]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[14]  David D. Cox,et al.  Untangling invariant object recognition , 2007, Trends in Cognitive Sciences.

[15]  Joachim Gross,et al.  Good practice for conducting and reporting MEG research , 2013, NeuroImage.

[16]  D. V. Essen,et al.  Neural mechanisms of form and motion processing in the primate visual system , 1994, Neuron.

[17]  Rufin Vogels,et al.  Stimulus features coded by single neurons of a macaque body category selective patch , 2016, Proceedings of the National Academy of Sciences.

[18]  K. Grill-Spector,et al.  The functional architecture of the ventral temporal cortex and its role in categorization , 2014, Nature Reviews Neuroscience.

[19]  Keiji Tanaka,et al.  Object category structure in response patterns of neuronal population in monkey inferior temporal cortex. , 2007, Journal of neurophysiology.

[20]  N. Sigala,et al.  Visual categorization shapes feature selectivity in the primate temporal cortex , 2002, Nature.

[21]  N. Kanwisher,et al.  Stages of processing in face perception: an MEG study , 2002, Nature Neuroscience.

[22]  Philippe G Schyns,et al.  RAP: a new framework for visual categorization , 2002, Trends in Cognitive Sciences.

[23]  Philippe G. Schyns,et al.  Efficient Information Contents Flow Down from Memory to Predict the Identity of Faces , 2017, bioRxiv.

[24]  Karl J. Friston Hierarchical Models in the Brain , 2008, PLoS Comput. Biol..

[25]  P. Sajda,et al.  Temporal characterization of the neural correlates of perceptual decision making in the human brain. , 2006, Cerebral cortex.

[26]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[27]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[28]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[29]  A. Tversky Features of Similarity , 1977 .

[30]  J. Gallant,et al.  Identifying natural images from human brain activity , 2008, Nature.

[31]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[32]  P. Schyns,et al.  Superstitious Perceptions Reveal Properties of Internal Representations , 2003, Psychological science.

[33]  Guillaume A. Rousselet,et al.  The Deceptively Simple N170 Reflects Network Information Processing Mechanisms Involving Visual Feature Coding and Transfer Across Hemispheres , 2016, bioRxiv.

[34]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[35]  P. Schyns,et al.  A principled method for determining the functionality of brain responses , 2003, Neuroreport.

[36]  Marcel A. J. van Gerven,et al.  Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream , 2014, The Journal of Neuroscience.

[37]  D. Schacter,et al.  A sensory signature that distinguishes true from false memories , 2004, Nature Neuroscience.

[38]  O. Sporns,et al.  Complex brain networks: graph theoretical analysis of structural and functional systems , 2009, Nature Reviews Neuroscience.

[39]  Antonio Torralba,et al.  Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence , 2016, Scientific Reports.

[40]  Philippe G Schyns,et al.  Diagnostic recognition: task constraints, object information, and their interactions , 1998, Cognition.

[41]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[42]  N. Kanwisher,et al.  The selectivity of the occipitotemporal M170 for faces , 2000, Neuroreport.

[43]  Doris Y. Tsao,et al.  The Code for Facial Identity in the Primate Brain , 2017, Cell.

[44]  K. Grill-Spector,et al.  The human visual cortex. , 2004, Annual review of neuroscience.

[45]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[46]  Guillaume A. Rousselet,et al.  A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula , 2016, bioRxiv.

[47]  M. A. MacIver,et al.  Neuroscience Needs Behavior: Correcting a Reductionist Bias , 2017, Neuron.

[48]  Liina Pylkkänen,et al.  A visual M170 effect of morphological complexity , 2009 .

[49]  Philippe G. Schyns,et al.  Dynamics of Trimming the Content of Face Representations for Categorization in the Brain , 2009, PLoS Comput. Biol..

[50]  Bruno Rossion,et al.  Parametric design and correlational analyses help integrating fMRI and electrophysiological data during face processing , 2004, NeuroImage.

[51]  Radoslaw Martin Cichy,et al.  Resolving human object recognition in space and time , 2014, Nature Neuroscience.

[52]  Vinh Thai Nguyen,et al.  The superior temporal sulcus and the N170 during face processing: Single trial analysis of concurrent EEG–fMRI , 2014, NeuroImage.

[53]  Caspar M. Schwiedrzik,et al.  High-Level Prediction Signals in a Low-Level Area of the Macaque Face-Processing Hierarchy , 2017, Neuron.

[54]  M. Bar The proactive brain: using analogies and associations to generate predictions , 2007, Trends in Cognitive Sciences.

[55]  D C Van Essen,et al.  Information processing in the primate visual system: an integrated systems perspective. , 1992, Science.

[56]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[57]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[58]  Bruno Rossion,et al.  Early lateralization and orientation tuning for face, word, and object processing in the visual cortex , 2003, NeuroImage.