Multivoxel Object Representations in Adult Human Visual Cortex Are Flexible: An Associative Learning Study

Learning associations between co-occurring events enables us to extract structure from our environment. Medial-temporal lobe structures are critical for associative learning. However, the role of the ventral visual pathway (VVP) in associative learning is not clear. Do multivoxel object representations in the VVP reflect newly formed associations? We show that VVP multivoxel representations become more similar to each other after human participants learn arbitrary new associations between pairs of unrelated objects (faces, houses, cars, chairs). Participants were scanned before and after 15 days of associative learning. To evaluate how object representations changed, a classifier was trained on discriminating two nonassociated categories (e.g., faces/houses) and tested on discriminating their paired associates (e.g., cars/chairs). Because the associations were arbitrary and counterbalanced across participants, there was initially no particular reason for this cross-classification decision to tend toward either alternative. Nonetheless, after learning, cross-classification performance increased in the VVP (but not hippocampus), on average by 3.3%, with some voxels showing increases of up to 10%. For example, a chair multivoxel representation that initially resembled neither face nor house representations was, after learning, classified as more similar to that of faces for participants who associated chairs with faces and to that of houses for participants who associated chairs with houses. Additionally, learning produced long-lasting perceptual consequences. In a behavioral priming experiment performed several months later, the change in cross-classification performance was correlated with the degree of priming. Thus, VVP multivoxel representations are not static but become more similar to each other after associative learning.

[1]  Y. Miyashita Neuronal correlate of visual associative long-term memory in the primate temporal cortex , 1988, Nature.

[2]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[3]  Andrew D. Straw,et al.  Vision Egg: an Open-Source Library for Realtime Visual Stimulus Generation , 2008, Frontiers Neuroinformatics.

[4]  Yasushi Miyashita,et al.  Forward Processing of Long-Term Associative Memory in Monkey Inferotemporal Cortex , 2003, The Journal of Neuroscience.

[5]  Y. Miyashita,et al.  Neural organization for the long-term memory of paired associates , 1991, Nature.

[6]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[7]  L. Reddy,et al.  Category selectivity in the ventral visual pathway confers robustness to clutter and diverted attention , 2010 .

[8]  H. Eichenbaum,et al.  Critical role of the hippocampus in memory for sequences of events , 2002, Nature Neuroscience.

[9]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[10]  N. Sigala,et al.  Visual categorization shapes feature selectivity in the primate temporal cortex , 2002, Nature.

[11]  N. Kanwisher,et al.  Can generic expertise explain special processing for faces? , 2007, Trends in Cognitive Sciences.

[12]  L. Squire,et al.  Neuronal representations of stimulus associations develop in the temporal lobe during learning , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Rankin W. McGugin,et al.  High-resolution imaging of expertise reveals reliable object selectivity in the fusiform face area related to perceptual performance , 2012, Proceedings of the National Academy of Sciences.

[14]  C. Baker,et al.  The neural basis of visual object learning , 2010, Trends in Cognitive Sciences.

[15]  Andrew P. Yonelinas,et al.  Perirhinal Cortex Supports Encoding and Familiarity-Based Recognition of Novel Associations , 2008, Neuron.

[16]  Stefan Pollmann,et al.  PyMVPA: a Python Toolbox for Multivariate Pattern Analysis of fMRI Data , 2009, Neuroinformatics.

[17]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[18]  Taketoshi Ono,et al.  Neural Correlates of Associative Face Memory in the Anterior Inferior Temporal Cortex of Monkeys , 2010, The Journal of Neuroscience.

[19]  Rachel A. Diana,et al.  Imaging recollection and familiarity in the medial temporal lobe: a three-component model , 2007, Trends in Cognitive Sciences.

[20]  I. Fried,et al.  Rapid Encoding of New Memories by Individual Neurons in the Human Brain , 2015, Neuron.

[21]  Johan Wagemans,et al.  Subordinate Categorization Enhances the Neural Selectivity in Human Object-selective Cortex for Fine Shape Differences , 2009, Journal of Cognitive Neuroscience.

[22]  K. Grill-Spector The neural basis of object perception , 2003, Current Opinion in Neurobiology.

[23]  D. V. van Essen,et al.  The Processing of Visual Shape in the Cerebral Cortex of Human and Nonhuman Primates: A Functional Magnetic Resonance Imaging Study , 2004, The Journal of Neuroscience.

[24]  Stephen Jose Hanson,et al.  High-resolution imaging of the fusiform face area (FFA) using multivariate non-linear classifiers shows diagnosticity for non-face categories , 2011, NeuroImage.

[25]  L. Davachi Item, context and relational episodic encoding in humans , 2006, Current Opinion in Neurobiology.

[26]  L. Frank,et al.  Single Neurons in the Monkey Hippocampus and Learning of New Associations , 2003, Science.

[27]  N. Kanwisher Domain specificity in face perception , 2000, Nature Neuroscience.

[28]  N. Kanwisher,et al.  The lateral occipital complex and its role in object recognition , 2001, Vision Research.

[29]  M. Tarr,et al.  Activation of the middle fusiform 'face area' increases with expertise in recognizing novel objects , 1999, Nature Neuroscience.

[30]  Leila Reddy,et al.  Coding of visual objects in the ventral stream , 2006, Current Opinion in Neurobiology.

[31]  Dwight J. Kravitz,et al.  Task context impacts visual object processing differentially across the cortex , 2014, Proceedings of the National Academy of Sciences.

[32]  S. Gerber,et al.  Unsupervised Natural Experience Rapidly Alters Invariant Object Representation in Visual Cortex , 2008 .

[33]  Thomas Serre,et al.  Reading the mind's eye: Decoding category information during mental imagery , 2010, NeuroImage.

[34]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[35]  Johan Wagemans,et al.  Distributed subordinate specificity for bodies, faces, and buildings in human ventral visual cortex , 2010, NeuroImage.

[36]  N. Kanwisher,et al.  Discrimination Training Alters Object Representations in Human Extrastriate Cortex , 2006, The Journal of Neuroscience.

[37]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[38]  Brice A. Kuhl,et al.  Successful Remembering Elicits Event-Specific Activity Patterns in Lateral Parietal Cortex , 2014, The Journal of Neuroscience.

[39]  Vaidehi S. Natu,et al.  Category-Specific Cortical Activity Precedes Retrieval During Memory Search , 2005, Science.

[40]  E. Miller,et al.  Experience-dependent sharpening of visual shape selectivity in inferior temporal cortex. , 2005, Cerebral cortex.

[41]  Lisa R. Betts,et al.  Distributed Neural Plasticity for Shape Learning in the Human Visual Cortex , 2005, PLoS biology.

[42]  Brice A. Kuhl,et al.  Fidelity of neural reactivation reveals competition between memories , 2011, Proceedings of the National Academy of Sciences.

[43]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[44]  T. Carlson,et al.  Patterns of Activity in the Categorical Representations of Objects , 2003 .

[45]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[46]  Heekyeong Park,et al.  Neural activity supporting the formation of associative memory versus source memory , 2012, Brain Research.

[47]  L. Reddy,et al.  Learning of anticipatory responses in single neurons of the human medial temporal lobe , 2015, Nature Communications.

[48]  Y. Miyashita,et al.  Neuronal correlate of pictorial short-term memory in the primate temporal cortexYasushi Miyashita , 1988, Nature.

[49]  Lauren V. Kustner,et al.  Shaping of Object Representations in the Human Medial Temporal Lobe Based on Temporal Regularities , 2012, Current Biology.

[50]  Bryan R. Conroy,et al.  A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex , 2011, Neuron.

[51]  Alexander Borst,et al.  How does Nature Program Neuron Types? , 2008, Front. Neurosci..

[52]  R. Desimone,et al.  Responses of Macaque Perirhinal Neurons during and after Visual Stimulus Association Learning , 1999, The Journal of Neuroscience.

[53]  Z. Kourtzi,et al.  Learning Alters the Tuning of Functional Magnetic Resonance Imaging Patterns for Visual Forms , 2010, The Journal of Neuroscience.

[54]  Marvin M. Chun,et al.  Neural Evidence of Statistical Learning: Efficient Detection of Visual Regularities Without Awareness , 2009, Journal of Cognitive Neuroscience.

[55]  Jack L. Gallant,et al.  Natural Scene Statistics Account for the Representation of Scene Categories in Human Visual Cortex , 2013, Neuron.

[56]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[57]  M. Riesenhuber,et al.  Categorization Training Results in Shape- and Category-Selective Human Neural Plasticity , 2007, Neuron.

[58]  C. Gross,et al.  Neural ensemble coding in inferior temporal cortex. , 1994, Journal of neurophysiology.

[59]  N. Kanwisher,et al.  A Cortical Area Selective for Visual Processing of the Human Body , 2001, Science.

[60]  Kenneth A. Norman,et al.  Recollection, Familiarity, and Cortical Reinstatement: A Multivoxel Pattern Analysis , 2009, Neuron.

[61]  M. Behrmann,et al.  Impact of learning on representation of parts and wholes in monkey inferotemporal cortex , 2002, Nature Neuroscience.

[62]  K. Grill-Spector,et al.  The human visual cortex. , 2004, Annual review of neuroscience.

[63]  Roberto Cabeza,et al.  Neural Correlates of Relational Memory: Successful Encoding and Retrieval of Semantic and Perceptual Associations , 2005, The Journal of Neuroscience.

[64]  Keiji Tanaka,et al.  Matching Categorical Object Representations in Inferior Temporal Cortex of Man and Monkey , 2008, Neuron.

[65]  Tolga Çukur,et al.  Functional Subdomains within Human FFA , 2013, The Journal of Neuroscience.

[66]  R. Clark,et al.  The medial temporal lobe. , 2004, Annual review of neuroscience.

[67]  S. Petersen,et al.  Memory's echo: vivid remembering reactivates sensory-specific cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[68]  A. Wagner,et al.  Cortical reinstatement mediates the relationship between content-specific encoding activity and subsequent recollection decisions. , 2014, Cerebral cortex.

[69]  E. Tulving,et al.  Reactivation of encoding-related brain activity during memory retrieval. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[70]  M. Bar,et al.  The parahippocampal cortex mediates spatial and nonspatial associations. , 2007, Cerebral cortex.

[71]  G. Rees,et al.  Predicting the orientation of invisible stimuli from activity in human primary visual cortex , 2005, Nature Neuroscience.

[72]  S. Edelman,et al.  Differential Processing of Objects under Various Viewing Conditions in the Human Lateral Occipital Complex , 1999, Neuron.

[73]  N. Kanwisher,et al.  How Distributed Is Visual Category Information in Human Occipito-Temporal Cortex? An fMRI Study , 2002, Neuron.