Tracking Your Mind's Eye during Recollection: Decoding the Long-Term Recall of Short Audiovisual Clips

Unlike familiarity, recollection involves the ability to reconstruct mentally previous events that results in a strong sense of reliving. According to the reinstatement hypothesis, this specific feature emerges from the reactivation of cortical patterns involved during information exposure. Over time, the retrieval of specific details becomes more difficult, and memories become increasingly supported by familiarity judgments. The multiple trace theory (MTT) explains the gradual loss of episodic details by a transformation in the memory representation, a view that is not shared by the standard consolidation model. In this study, we tested the MTT in light of the reinstatement hypothesis. The temporal dynamics of mental imagery from long-term memory were investigated and tracked over the passage of time. Participant EEG activity was recorded during the recall of short audiovisual clips that had been watched 3 weeks, 1 day, or a few hours beforehand. The recall of the audiovisual clips was assessed using a Remember/Know/New procedure, and snapshots of clips were used as recall cues. The decoding matrices obtained from the multivariate pattern analyses revealed sustained patterns that occurred at long latencies (>500 msec poststimulus onset) that faded away over the retention intervals and that emerged from the same neural processes. Overall, our data provide further evidence toward the MTT and give new insights into the exploration of our “mind's eye.”

[1]  Jennifer S. Burt,et al.  What do students remember? Episodic memory and the development of schematization , 2004 .

[2]  J. Dunn Remember-know: a matter of confidence. , 2004, Psychological review.

[3]  J. Wixted Remember/Know judgments in cognitive neuroscience: An illustration of the underrepresented point of view. , 2009, Learning & memory.

[4]  Daniel L. Schacter,et al.  ERP correlates of recognition memory: Effects of retention interval and false alarms , 2006, Brain Research.

[5]  D. Hassabis,et al.  Decoding Individual Episodic Memory Traces in the Human Hippocampus , 2010, Current Biology.

[6]  William F. Brewer,et al.  Reports of Mental Imagery in Retrieval from Long-Term Memory , 1996, Consciousness and Cognition.

[7]  Dennis Norris,et al.  Perception as evidence accumulation and Bayesian inference: insights from masked priming. , 2008, Journal of experimental psychology. General.

[8]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[9]  Pascale Piolino,et al.  Episodic autobiographical memories over the course of time: Cognitive, neuropsychological and neuroimaging findings , 2009, Neuropsychologia.

[10]  Michael D. Rugg,et al.  Content-specificity of the neural correlates of recollection , 2005, Neuropsychologia.

[11]  Mark C. W. van Rossum,et al.  Systematic biases in early ERP and ERF components as a result of high-pass filtering , 2012, Journal of Neuroscience Methods.

[12]  Simon J Thorpe Grandmother Cells and Distributed Representations , 2010 .

[13]  Wael El-Deredy,et al.  Old-new ERP effects and remote memories: the late parietal effect is absent as recollection fails whereas the early mid-frontal effect persists as familiarity is retained , 2015, Front. Hum. Neurosci..

[14]  Kenneth A. Norman,et al.  Recollection, Familiarity, and Cortical Reinstatement: A Multivoxel Pattern Analysis , 2009, Neuron.

[15]  Jared F. Danker,et al.  The ghosts of brain states past: remembering reactivates the brain regions engaged during encoding. , 2010, Psychological bulletin.

[16]  M. Rugg,et al.  An event-related potential study of recognition memory with and without retrieval of source. , 1996, Brain : a journal of neurology.

[17]  M. Shadlen,et al.  Decision Making and Sequential Sampling from Memory , 2016, Neuron.

[18]  John T Wixted,et al.  Rethinking Familiarity: Remember/Know Judgments in Free Recall. , 2013, Journal of memory and language.

[19]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[20]  M. Rugg,et al.  Context Effects on the Neural Correlates of Recognition Memory An Electrophysiological Study , 2001, Neuron.

[21]  E. Wilding In what way does the parietal ERP old/new effect index recollection? , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[22]  P. Frankland,et al.  The organization of recent and remote memories , 2005, Nature Reviews Neuroscience.

[23]  Michael D. Rugg,et al.  Decoding the content of recollection within the core recollection network and beyond , 2017, Cortex.

[24]  G. Buzsáki Two-stage model of memory trace formation: A role for “noisy” brain states , 1989, Neuroscience.

[25]  Floris P. de Lange,et al.  Differential temporal dynamics during visual imagery and perception , 2017, bioRxiv.

[26]  D. Tsivilis,et al.  The electrophysiological correlates of recent and remote recollection , 2013, Neuropsychologia.

[27]  M. Khamassi,et al.  Replay of rule-learning related neural patterns in the prefrontal cortex during sleep , 2009, Nature Neuroscience.

[28]  Douglas L. Hintzman,et al.  Judgments of frequency and recognition memory in a multiple-trace memory model. , 1988 .

[29]  J. Csicsvari,et al.  Communication between neocortex and hippocampus during sleep in rodents , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Gui Xue,et al.  Transformed Neural Pattern Reinstatement during Episodic Memory Retrieval , 2017, The Journal of Neuroscience.

[31]  A. Damasio Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition , 1989, Cognition.

[32]  R. Knight,et al.  Dissociable neural correlates for familiarity and recollection during the encoding and retrieval of pictures. , 2004, Brain research. Cognitive brain research.

[33]  G. Mandler Recognizing: The judgment of previous occurrence. , 1980 .

[34]  E. Tulving,et al.  Event-related brain potential correlates of two states of conscious awareness in memory. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[35]  A. Baddeley The episodic buffer: a new component of working memory? , 2000, Trends in Cognitive Sciences.

[36]  Jeffrey D. Johnson,et al.  Encoding-retrieval overlap in human episodic memory: a functional neuroimaging perspective. , 2008, Progress in brain research.

[37]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[38]  S. Dehaene Conscious and Nonconscious Processes:Distinct Forms of Evidence Accumulation? , 2011 .

[39]  A. Ishai,et al.  Distributed neural systems for the generation of visual images , 2000, NeuroImage.

[40]  J. Andel Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[41]  Brigitte Landeau,et al.  The Hippocampus Remains Activated over the Long Term for the Retrieval of Truly Episodic Memories , 2012, PloS one.

[42]  Karl J. Friston,et al.  Distinct Top-down and Bottom-up Brain Connectivity During Visual Perception and Imagery , 2017, Scientific Reports.

[43]  E. Tulving,et al.  Episodic and semantic memory , 1972 .

[44]  Sébastien M. Crouzet,et al.  Taste Quality Decoding Parallels Taste Sensations , 2015, Current Biology.

[45]  R. Shiffrin,et al.  A model for recognition memory: REM—retrieving effectively from memory , 1997, Psychonomic bulletin & review.

[46]  E. Tulving Memory and consciousness. , 1985 .

[47]  S. Petersen,et al.  Memory's echo: vivid remembering reactivates sensory-specific cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[48]  E. Tulving,et al.  Reactivation of encoding-related brain activity during memory retrieval. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[49]  A Koriat,et al.  The combined contributions of the cue-familiarity and accessibility heuristics to feelings of knowing. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[50]  G. Winocur,et al.  Remembering our past: functional neuroanatomy of recollection of recent and very remote personal events. , 2004, Cerebral cortex.

[51]  T. Curran Brain potentials of recollection and familiarity , 2000, Memory & cognition.

[52]  Michael D. Rugg,et al.  The relationship between electrophysiological correlates of recollection and amount of information retrieved , 2006, Brain Research.

[53]  Karl J. Friston,et al.  Where bottom-up meets top-down: neuronal interactions during perception and imagery. , 2004, Cerebral cortex.

[54]  Elisabeth J. Ploran,et al.  Evidence Accumulation and the Moment of Recognition: Dissociating Perceptual Recognition Processes Using fMRI , 2007, The Journal of Neuroscience.

[55]  E. Tulving Elements of episodic memory , 1983 .

[56]  Simon Thorpe,et al.  Local vs. Distributed Coding , 1989 .

[57]  Brice A. Kuhl,et al.  Successful Remembering Elicits Event-Specific Activity Patterns in Lateral Parietal Cortex , 2014, The Journal of Neuroscience.

[58]  Eleanor A. Maguire,et al.  Two years later – Revisiting autobiographical memory representations in vmPFC and hippocampus , 2017, Neuropsychologia.

[59]  Benjamin J. Shannon,et al.  Parietal lobe contributions to episodic memory retrieval , 2005, Trends in Cognitive Sciences.

[60]  J M Gardiner,et al.  Changes in memory awareness during learning: the acquisition of knowledge by psychology undergraduates. , 1997, Journal of experimental psychology. General.

[61]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[62]  Tim Curran,et al.  ERP old/new effects at different retention intervals in recency discrimination tasks. , 2004, Brain research. Cognitive brain research.

[63]  R N Henson,et al.  Activity in prefrontal cortex, not hippocampus, varies parametrically with the increasing remoteness of memories , 2001, Neuroreport.

[64]  M. Rugg,et al.  Event-related potentials and recognition memory , 2007, Trends in Cognitive Sciences.

[65]  A. Yonelinas The Nature of Recollection and Familiarity: A Review of 30 Years of Research , 2002 .

[66]  Nicco Reggente,et al.  Decoding fMRI Signatures of Real-world Autobiographical Memory Retrieval , 2016, Journal of Cognitive Neuroscience.

[67]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[68]  J. Eich A composite holographic associative recall model. , 1982 .

[69]  Axel Mecklinger,et al.  ERP correlates of true and false recognition after different retention delays: stimulus- and response-related processes. , 2003, Psychophysiology.

[70]  Eleanor A Maguire,et al.  Lateral Asymmetry in the Hippocampal Response to the Remoteness of Autobiographical Memories , 2003, The Journal of Neuroscience.

[71]  Richard C. Atkinson,et al.  Recognition time for information stored in long-term memory , 1971 .

[72]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[73]  Leslie G. Ungerleider,et al.  Distributed Neural Systems for the Generation of Visual Images , 2000, Neuron.

[74]  A. Richardson-Klavehn,et al.  Remembering and knowing , 2000 .

[75]  Anthony D. Wagner,et al.  Detecting individual memories through the neural decoding of memory states and past experience , 2010, Proceedings of the National Academy of Sciences.

[76]  Kaia L. Vilberg,et al.  Memory retrieval and the parietal cortex: A review of evidence from a dual-process perspective , 2008, Neuropsychologia.

[77]  C. Moulin,et al.  An active inference and epistemic value view of metacognition , 2015, Cognitive neuroscience.

[78]  G. Winocur,et al.  Memory Transformation and Systems Consolidation , 2011, Journal of the International Neuropsychological Society.

[79]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[80]  Jeffrey D. Johnson,et al.  Recollection and the reinstatement of encoding-related cortical activity. , 2007, Cerebral cortex.

[81]  Maxime Cauchoix,et al.  Fast ventral stream neural activity enables rapid visual categorization , 2015 .

[82]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[83]  Rufin VanRullen,et al.  Four Common Conceptual Fallacies in Mapping the Time Course of Recognition , 2011, Front. Psychology.

[84]  James L. McClelland,et al.  Familiarity breeds differentiation: a subjective-likelihood approach to the effects of experience in recognition memory. , 1998, Psychological review.

[85]  L. Squire,et al.  Retrograde amnesia and memory consolidation: a neurobiological perspective , 1995, Current Opinion in Neurobiology.

[86]  Kaia L. Vilberg,et al.  Brain Networks Underlying Episodic Memory Retrieval This Review Comes from a Themed Issue on Macrocircuits Memory Signals within the Mtl , 2022 .

[87]  G. Buzsáki,et al.  Selective suppression of hippocampal ripples impairs spatial memory , 2009, Nature Neuroscience.

[88]  Eleanor A Maguire,et al.  Representations of recent and remote autobiographical memories in hippocampal subfields , 2013, Hippocampus.

[89]  Nikolaus Weiskopf,et al.  Detecting Representations of Recent and Remote Autobiographical Memories in vmPFC and Hippocampus , 2012, The Journal of Neuroscience.