Multi-voxel patterns of visual category representation during episodic encoding are predictive of subsequent memory

Successful encoding of episodic memories is thought to depend on contributions from prefrontal and temporal lobe structures. Neural processes that contribute to successful encoding have been extensively explored through univariate analyses of neuroimaging data that compare mean activity levels elicited during the encoding of events that are subsequently remembered vs. those subsequently forgotten. Here, we applied pattern classification to fMRI data to assess the degree to which distributed patterns of activity within prefrontal and temporal lobe structures elicited during the encoding of word-image pairs were diagnostic of the visual category (Face or Scene) of the encoded image. We then assessed whether representation of category information was predictive of subsequent memory. Classification analyses indicated that temporal lobe structures contained information robustly diagnostic of visual category. Information in prefrontal cortex was less diagnostic of visual category, but was nonetheless associated with highly reliable classifier-based evidence for category representation. Critically, trials associated with greater classifier-based estimates of category representation in temporal and prefrontal regions were associated with a higher probability of subsequent remembering. Finally, consideration of trial-by-trial variance in classifier-based measures of category representation revealed positive correlations between prefrontal and temporal lobe representations, with the strength of these correlations varying as a function of the category of image being encoded. Together, these results indicate that multi-voxel representations of encoded information can provide unique insights into how visual experiences are transformed into episodic memories.

[1]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[2]  Nancy Kanwisher,et al.  A cortical representation of the local visual environment , 1998, Nature.

[3]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[4]  Kenneth A. Norman,et al.  Multivoxel Pattern Analysis Reveals Increased Memory Targeting and Reduced Use of Retrieved Details during Single-Agenda Source Monitoring , 2009, The Journal of Neuroscience.

[5]  C. Grady,et al.  Event-related fMRI studies of episodic encoding and retrieval: Meta-analyses using activation likelihood estimation , 2009, Neuropsychologia.

[6]  Mark D'Esposito,et al.  The Prefrontal Cortex Modulates Category Selectivity in Human Extrastriate Cortex , 2011, Journal of Cognitive Neuroscience.

[7]  A. Dale,et al.  Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. , 1998, Science.

[8]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[9]  A P Shimamura,et al.  Memory and the Prefrontal Cortex a , 1995, Annals of the New York Academy of Sciences.

[10]  Kalanit Grill-Spector,et al.  Sparsely-distributed organization of face and limb activations in human ventral temporal cortex , 2010, NeuroImage.

[11]  M. Posner,et al.  On the genesis of abstract ideas. , 1968, Journal of experimental psychology.

[12]  A. Ishai,et al.  Recollection- and Familiarity-Based Decisions Reflect Memory Strength , 2008, Frontiers in systems neuroscience.

[13]  A. Wagner,et al.  Sources of Method Bias in Social Science Research and Recommendations on How to Control It , 2015 .

[14]  Earl K. Miller,et al.  Selective representation of relevant information by neurons in the primate prefrontal cortex , 1998, Nature.

[15]  Mark G. Baxter,et al.  Functional localization within the prefrontal cortex: missing the forest for the trees? , 2010, Trends in Neurosciences.

[16]  Y. Miyashita,et al.  Top-down signal from prefrontal cortex in executive control of memory retrieval , 1999, Nature.

[17]  J. Grafman,et al.  Human prefrontal cortex: processing and representational perspectives , 2003, Nature Reviews Neuroscience.

[18]  Kenneth A. Norman,et al.  Recollection, Familiarity, and Cortical Reinstatement: A Multivoxel Pattern Analysis , 2009, Neuron.

[19]  J. Duncan An adaptive coding model of neural function in prefrontal cortex , 2001 .

[20]  Gary H. Glover,et al.  High-resolution fMRI of Content-sensitive Subsequent Memory Responses in Human Medial Temporal Lobe , 2010, Journal of Cognitive Neuroscience.

[21]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[22]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[23]  Activity Precedes Retrieval During Memory Search , 2005 .

[24]  L. Squire Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory , 1992, Journal of Cognitive Neuroscience.

[25]  Conny F. Schmidt,et al.  Face perception is mediated by a distributed cortical network , 2005, Brain Research Bulletin.

[26]  Jemett L. Desmond,et al.  Semantic encoding and retrieval in the left inferior prefrontal cortex: a functional MRI study of task difficulty and process specificity , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  E. Tulving,et al.  Frontal lobe damage produces episodic memory impairment , 1995, Journal of the International Neuropsychological Society.

[28]  M. Rugg,et al.  Task-dependency of the neural correlates of episodic encoding as measured by fMRI. , 2001, Cerebral cortex.

[29]  J. Desmond,et al.  Material-specific lateralization in the medial temporal lobe and prefrontal cortex during memory encoding. , 2001, Brain : a journal of neurology.

[30]  Lee Ryan,et al.  The effect of scene context on episodic object recognition: Parahippocampal cortex mediates memory encoding and retrieval success , 2007, Hippocampus.

[31]  S C Rao,et al.  Integration of what and where in the primate prefrontal cortex. , 1997, Science.

[32]  J. D. E. Gabrieli,et al.  Integration of diverse information in working memory within the frontal lobe , 2000, Nature Neuroscience.

[33]  K. Paller,et al.  Observing the transformation of experience into memory , 2002, Trends in Cognitive Sciences.

[34]  D. Schacter,et al.  Interactions Between Forms of Memory: When Priming Hinders New Episodic Learning , 2000, Journal of Cognitive Neuroscience.

[35]  Charan Ranganath,et al.  Prefrontal Cortex and Long-Term Memory Encoding: An Integrative Review of Findings from Neuropsychology and Neuroimaging , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[36]  Jennifer A. Mangels,et al.  Neocortical Connectivity during Episodic Memory Formation , 2006, PLoS biology.

[37]  A R McIntosh,et al.  Neural correlates of the episodic encoding of pictures and words. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[38]  Anthony D. Wagner,et al.  Neural Priming in Human Frontal Cortex: Multiple Forms of Learning Reduce Demands on the Prefrontal Executive System , 2009, Journal of Cognitive Neuroscience.

[39]  H. Spiers,et al.  Prefrontal and medial temporal lobe interactions in long-term memory , 2003, Nature Reviews Neuroscience.

[40]  P. Goldman-Rakic,et al.  Areal segregation of face-processing neurons in prefrontal cortex. , 1997, Science.

[41]  R. Buckner,et al.  Neural correlates of verbal memory encoding during semantic and structural processing tasks , 2001, Neuroreport.

[42]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[43]  C. Stern,et al.  Prefrontal–Temporal Circuitry for Episodic Encoding and Subsequent Memory , 2000, The Journal of Neuroscience.

[44]  Theodore P. Zanto,et al.  Causal role of the prefrontal cortex in top-down modulation of visual processing and working memory , 2011, Nature Neuroscience.

[45]  Adam Gazzaley,et al.  Measuring functional connectivity during distinct stages of a cognitive task , 2004, NeuroImage.

[46]  Stephen D. Mayhew,et al.  Article Learning Shapes the Representation of Behavioral Choice in the Human Brain , 2022 .

[47]  John D E Gabrieli,et al.  Working memory and long‐term memory for faces: Evidence from fMRI and global amnesia for involvement of the medial temporal lobes , 2006, Hippocampus.

[48]  Hongkeun Kim,et al.  Neural activity that predicts subsequent memory and forgetting: A meta-analysis of 74 fMRI studies , 2011, NeuroImage.

[49]  Michael B. Miller,et al.  Separable Routes to Human Memory Formation: Dissociating Task and Material Contributions in the Prefrontal Cortex , 2004, Journal of Cognitive Neuroscience.

[50]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[51]  J. Desmond,et al.  Material‐specific lateralization of prefrontal activation during episodic encoding and retrieval , 1998, Neuroreport.

[52]  W. Scoville,et al.  LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS , 1957, Journal of neurology, neurosurgery, and psychiatry.

[53]  J. Mumford,et al.  Greater Neural Pattern Similarity Across Repetitions Is Associated with Better Memory , 2010, Science.

[54]  Doris Y. Tsao,et al.  Patches of face-selective cortex in the macaque frontal lobe , 2008, Nature Neuroscience.

[55]  Steven E. Prince,et al.  Encoding and retrieving faces and places: Distinguishing process- and stimulus-specific differences in brain activity , 2009, Neuropsychologia.

[56]  Jarrod A. Lewis-Peacock,et al.  Temporary Activation of Long-Term Memory Supports Working Memory , 2008, The Journal of Neuroscience.

[57]  A. Ishai,et al.  The Gender of Face Stimuli is Represented in Multiple Regions in the Human Brain , 2011, Front. Hum. Neurosci..

[58]  H. Eichenbaum,et al.  Memory, amnesia, and the hippocampal system , 1993 .

[59]  J. Desmond,et al.  Making memories: brain activity that predicts how well visual experience will be remembered. , 1998, Science.

[60]  Karl J. Friston,et al.  Functional Connectivity: The Principal-Component Analysis of Large (PET) Data Sets , 1993, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[61]  Randy L. Buckner,et al.  Set-and Code-Specific Activation in the Frontal Cortex: An fMRI Study of Encoding and Retrieval of Faces and Words , 1999, Journal of Cognitive Neuroscience.

[62]  Vaidehi S. Natu,et al.  Category-Specific Cortical Activity Precedes Retrieval During Memory Search , 2005, Science.

[63]  Anthony D. Wagner,et al.  Detecting individual memories through the neural decoding of memory states and past experience , 2010, Proceedings of the National Academy of Sciences.

[64]  M. Chun,et al.  Linking Implicit and Explicit Memory: Common Encoding Factors and Shared Representations , 2006, Neuron.

[65]  Koji Jimura,et al.  Prediction of subsequent recognition performance using brain activity in the medial temporal lobe , 2011, NeuroImage.

[66]  Brice A. Kuhl,et al.  Fidelity of neural reactivation reveals competition between memories , 2011, Proceedings of the National Academy of Sciences.

[67]  David J. Freedman,et al.  A Comparison of Primate Prefrontal and Inferior Temporal Cortices during Visual Categorization , 2003, The Journal of Neuroscience.

[68]  L. Davachi,et al.  Content-specific source encoding in the human medial temporal lobe. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[69]  Martin Lepage,et al.  A face to remember: emotional expression modulates prefrontal activity during memory formation , 2005, NeuroImage.

[70]  Leslie G. Ungerleider,et al.  An area specialized for spatial working memory in human frontal cortex. , 1998, Science.

[71]  P. Goldman-Rakic,et al.  Dissociation of object and spatial processing domains in primate prefrontal cortex. , 1993, Science.

[72]  Anthony D Wagner,et al.  Assembling and encoding word representations: fMRI subsequent memory effects implicate a role for phonological control , 2003, Neuropsychologia.

[73]  M. Giese,et al.  Flexible Coding for Categorical Decisions in the Human Brain , 2007, The Journal of Neuroscience.

[74]  C. Ranganath,et al.  Prefrontal and Medial Temporal Lobe Activity at Encoding Predicts Temporal Context Memory , 2010, The Journal of Neuroscience.

[75]  R. Buckner,et al.  Functional-Anatomic Correlates of Individual Differences in Memory , 2006, Neuron.