Enhancing the Ecological Validity of fMRI Memory Research Using Virtual Reality

Functional magnetic resonance imaging (fMRI) is a powerful research tool to understand the neural underpinnings of human memory. However, as memory is known to be context-dependent, differences in contexts between naturalistic settings and the MRI scanner environment may potentially confound neuroimaging findings. Virtual reality (VR) provides a unique opportunity to mitigate this issue by allowing memories to be formed and/or retrieved within immersive, navigable, visuospatial contexts. This can enhance the ecological validity of task paradigms, while still ensuring that researchers maintain experimental control over critical aspects of the learning and testing experience. This mini-review surveys the growing body of fMRI studies that have incorporated VR to address critical questions about human memory. These studies have adopted a variety of approaches, including presenting research participants with VR experiences in the scanner, asking participants to retrieve information that they had previously acquired in a VR environment, or identifying neural correlates of behavioral metrics obtained through VR-based tasks performed outside the scanner. Although most such studies to date have focused on spatial or navigational memory, we also discuss the promise of VR in aiding other areas of memory research and facilitating research into clinical disorders.

[1]  Elizabeth R. Chrastil,et al.  There and Back Again: Hippocampus and Retrosplenial Cortex Track Homing Distance during Human Path Integration , 2015, The Journal of Neuroscience.

[2]  Damiaan Denys,et al.  Virtual Reality Objectifies the Diagnosis of Psychiatric Disorders: A Literature Review , 2017, Front. Psychiatry.

[3]  Bernhard E. Riecke,et al.  Moving through virtual reality without moving? , 2012, Cognitive Processing.

[4]  J. Jacobs Hippocampal theta oscillations are slower in humans than in rodents: implications for models of spatial navigation and memory , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[5]  D. Pennell,et al.  Functional abnormalities in normally appearing athletes following mild traumatic brain injury: a functional MRI study , 2010, Experimental Brain Research.

[6]  E. Maguire,et al.  The Well-Worn Route and the Path Less Traveled Distinct Neural Bases of Route Following and Wayfinding in Humans , 2003, Neuron.

[7]  Doug A. Bowman,et al.  Virtual Reality: How Much Immersion Is Enough? , 2007, Computer.

[8]  Robert W. Mitchell,et al.  Spatial cognition, spatial perception : mapping the self and space , 2010 .

[9]  H. T. Blair,et al.  Putting Fear in Its Place: Remapping of Hippocampal Place Cells during Fear Conditioning , 2004, The Journal of Neuroscience.

[10]  Dylan M. Jones,et al.  Navigating Buildings in "Desk-Top" Virtual Environments: Experimental Investigations Using Extended Navigational Experience , 1997 .

[11]  Arne D. Ekstrom,et al.  Impairments in precision, rather than spatial strategy, characterize performance on the virtual Morris Water Maze: A case study , 2016, Neuropsychologia.

[12]  Jason B. Mattingley,et al.  Negative Emotional Experiences during Navigation Enhance Parahippocampal Activity during Recall of Place Information , 2014, Journal of Cognitive Neuroscience.

[13]  Timothy E. J. Behrens,et al.  Organizing conceptual knowledge in humans with a gridlike code , 2016, Science.

[14]  Paul Pauli,et al.  Does pre-exposure inhibit fear context conditioning? A Virtual Reality Study , 2012, Journal of Neural Transmission.

[15]  E. Maguire,et al.  A Temporoparietal and Prefrontal Network for Retrieving the Spatial Context of Lifelike Events , 2001, NeuroImage.

[16]  Steven M. Smith Environmental context—dependent memory. , 1988 .

[17]  Adrian W. Gilmore,et al.  Are There Multiple Kinds of Episodic Memory? An fMRI Investigation Comparing Autobiographical and Recognition Memory Tasks , 2017, The Journal of Neuroscience.

[18]  S. Swinnen,et al.  Virtual water maze learning in human increases functional connectivity between posterior hippocampus and dorsal caudate , 2015, Human brain mapping.

[19]  D. Hassabis,et al.  Decoding Neuronal Ensembles in the Human Hippocampus , 2009, Current Biology.

[20]  Neil Burgess,et al.  Grid-like Processing of Imagined Navigation , 2016, Current Biology.

[21]  Bob G. Witmer,et al.  Judging Perceived and Traversed Distance in Virtual Environments , 1998, Presence.

[22]  Jesse Rissman,et al.  Multi-voxel pattern classification differentiates personally experienced event memories from secondhand event knowledge , 2018, NeuroImage.

[23]  Mayank R Mehta,et al.  Impaired spatial selectivity and intact phase precession in two-dimensional virtual reality , 2014, Nature Neuroscience.

[24]  Neil Burgess,et al.  Hippocampal Attractor Dynamics Predict Memory-Based Decision Making , 2016, Current Biology.

[25]  Misun Kim,et al.  Multivoxel Pattern Analysis Reveals 3D Place Information in the Human Hippocampus , 2017, The Journal of Neuroscience.

[26]  Christian F. Doeller,et al.  Evidence for grid cells in a human memory network , 2010, Nature.

[27]  John A. King,et al.  How vision and movement combine in the hippocampal place code , 2012, Proceedings of the National Academy of Sciences.

[28]  Hanspeter A. Mallot,et al.  Navigation and Acquisition of Spatial Knowledge in a Virtual Maze , 1998, Journal of Cognitive Neuroscience.

[29]  Colin Hawco,et al.  Source retrieval is not properly differentiated from object retrieval in early schizophrenia: An fMRI study using virtual reality , 2014, NeuroImage: Clinical.

[30]  John A. King,et al.  Human hippocampus and viewpoint dependence in spatial memory , 2002, Hippocampus.

[31]  Corey J. Bohil,et al.  Virtual reality in neuroscience research and therapy , 2011, Nature Reviews Neuroscience.

[32]  Evelyne Balteau,et al.  Sleep Modulates the Neural Substrates of Both Spatial and Contextual Memory Consolidation , 2008, PloS one.

[33]  R. Morris Developments of a water-maze procedure for studying spatial learning in the rat , 1984, Journal of Neuroscience Methods.

[34]  A. Baddeley,et al.  Context-dependent memory in two natural environments: on land and underwater. , 1975 .

[35]  Robert S. Astur,et al.  Factors affecting the hippocampal BOLD response during spatial memory , 2008, Behavioural Brain Research.

[36]  Gabriele Janzen,et al.  Neural Encoding of Objects Relevant for Navigation and Resting State Correlations with Navigational Ability , 2011, Journal of Cognitive Neuroscience.

[37]  Jonas Persson,et al.  Behavioural Brain Research , 2013 .

[38]  R. Golledge Wayfinding Behavior: Cognitive Mapping and Other Spatial Processes , 2010 .

[39]  Cornelis G. Weststeijn,et al.  Neural representation of object location and route direction: An event-related fMRI study , 2007, Brain Research.

[40]  Tobias Navarro Schröder,et al.  Grid-cell representations in mental simulation , 2016, eLife.

[41]  Steve Whittaker,et al.  Navigating through digital folders uses the same brain structures as real world navigation , 2015, Scientific Reports.

[42]  Halle R. Dimsdale-Zucker,et al.  CA1 and CA3 differentially support spontaneous retrieval of episodic contexts within human hippocampal subfields , 2017, bioRxiv.

[43]  Sarah Sharples,et al.  Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems , 2008, Displays.

[44]  J. Eriksson,et al.  Memory Self-Efficacy Beliefs Modulate Brain Activity when Encoding Real-World Future Intentions , 2013, PloS one.

[45]  Nathaniel J. Killian,et al.  Grid cells map the visual world , 2017, Nature Neuroscience.

[46]  Jian Xu,et al.  The Anterior Hippocampus Supports a Coarse, Global Environmental Representation and the Posterior Hippocampus Supports Fine-grained, Local Environmental Representations , 2013, Journal of Cognitive Neuroscience.

[47]  David S. Olton,et al.  Spatial memory and radial arm maze performance of rats , 1977 .

[48]  G. Davies,et al.  Memory in context : context in memory , 1990 .

[49]  Ellen M. Migo,et al.  Investigating virtual reality navigation in amnestic mild cognitive impairment using fMRI , 2016, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[50]  D. Tank,et al.  Membrane potential dynamics of grid cells , 2013, Nature.

[51]  Thackery I. Brown,et al.  A High‐resolution study of hippocampal and medial temporal lobe correlates of spatial context and prospective overlapping route memory , 2014, Hippocampus.

[52]  É. Sorita,et al.  The contribution of virtual reality to the diagnosis of spatial navigation disorders and to the study of the role of navigational aids: A systematic literature review. , 2017, Annals of physical and rehabilitation medicine.

[53]  B. Rothbaum,et al.  Virtual Reality-Enhanced Extinction of Phobias and Post-Traumatic Stress , 2017, Neurotherapeutics.

[54]  Kathleen B McDermott,et al.  Two types of event memory , 2013, Proceedings of the National Academy of Sciences.

[55]  Arne D. Ekstrom,et al.  Human Hippocampal CA1 Involvement during Allocentric Encoding of Spatial Information , 2009, The Journal of Neuroscience.

[56]  Arne D. Ekstrom,et al.  Dissociations within human hippocampal subregions during encoding and retrieval of spatial information , 2011, Hippocampus.

[57]  Jason B. Mattingley,et al.  Medial Parietal Cortex Encodes Perceived Heading Direction in Humans , 2010, The Journal of Neuroscience.

[58]  Christian F. Doeller,et al.  Interaction Between Hippocampus and Cerebellum Crus I in Sequence-Based but not Place-Based Navigation. , 2015, Cerebral cortex.

[59]  E. Maguire,et al.  Neurodevelopmental Aspects of Spatial Navigation: A Virtual Reality fMRI Study , 2002, NeuroImage.

[60]  Dmitriy Aronov,et al.  Engagement of Neural Circuits Underlying 2D Spatial Navigation in a Rodent Virtual Reality System , 2014, Neuron.

[61]  K G Thomas,et al.  Place learning in virtual space III: Investigation of spatial navigation training procedures and their application to fMRI and clinical neuropsychology , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[62]  Lukas Kunz,et al.  Reduced grid-cell–like representations in adults at genetic risk for Alzheimer’s disease , 2015, Science.

[63]  C. Hawco,et al.  Functional magnetic resonance imaging study of external source memory and its relation to cognitive insight in non‐clinical subjects , 2014, Psychiatry and clinical neurosciences.

[64]  Russell A. Epstein,et al.  Neural correlates of real-world route learning , 2010, NeuroImage.

[65]  Graciela Lara López,et al.  Navigation in Virtual Reality , 2020 .

[66]  G. Buzsáki,et al.  Memory, navigation and theta rhythm in the hippocampal-entorhinal system , 2013, Nature Neuroscience.

[67]  Markus Plank,et al.  Resting-State fMRI Activity Predicts Unsupervised Learning and Memory in an Immersive Virtual Reality Environment , 2014, PloS one.

[68]  Jesse Rissman,et al.  Neurocognitive mechanisms of real‐world autobiographical memory retrieval: insights from studies using wearable camera technology , 2017, Annals of the New York Academy of Sciences.

[69]  Jason B. Mattingley,et al.  Dissociable Representations of Environmental Size and Complexity in the Human Hippocampus , 2013, The Journal of Neuroscience.

[70]  Jian Xu,et al.  Persistent posterior and transient anterior medial temporal lobe activity during navigation , 2010, NeuroImage.

[71]  Arne D. Ekstrom,et al.  Complementary Roles of Human Hippocampal Subfields in Differentiation and Integration of Spatial Context , 2015, Journal of Cognitive Neuroscience.

[72]  Arne D. Ekstrom,et al.  Human hippocampal theta activity during virtual navigation , 2005, Hippocampus.

[73]  Arne D Ekstrom,et al.  Successful retrieval of competing spatial environments in humans involves hippocampal pattern separation mechanisms , 2015, eLife.

[74]  Alain Berthoz,et al.  Role of the human retrosplenial cortex/parieto-occipital sulcus in perspective priming , 2016, NeuroImage.

[75]  Jeffrey S. Taube,et al.  Is Navigation in Virtual Reality with fMRI Really Navigation? , 2013, Journal of Cognitive Neuroscience.

[76]  2D:4D and spatial abilities: From rats to humans , 2017 .

[77]  Lars Nyberg,et al.  Out-of-body–induced hippocampal amnesia , 2014, Proceedings of the National Academy of Sciences.

[78]  D. Selkoe Alzheimer's disease. , 2011, Cold Spring Harbor perspectives in biology.

[79]  A. Bartels,et al.  Parietal Cortex Codes for Egocentric Space beyond the Field of View , 2012, Current Biology.

[80]  P. Rodríguez,et al.  Neural decoding of goal locations in spatial navigation in humans with fMRI , 2009, Human brain mapping.

[81]  Zhishun Wang,et al.  Reward-Based Spatial Learning in Teens With Bulimia Nervosa. , 2016, Journal of the American Academy of Child and Adolescent Psychiatry.

[82]  Valerie A. Carr,et al.  Prospective representation of navigational goals in the human hippocampus , 2016, Science.

[83]  C. Harvey,et al.  Neuroscience: Virtual reality explored , 2016, Nature.

[84]  C. Stark,et al.  Pattern separation in the hippocampus , 2011, Trends in Neurosciences.

[85]  Walter Schneider,et al.  A Virtual Reality System for Neurobehavioral and Functional MRI Studies , 2003, Cyberpsychology Behav. Soc. Netw..

[86]  Zahra M. Aghajan,et al.  Theta Oscillations in the Human Medial Temporal Lobe during Real-World Ambulatory Movement , 2016, Current Biology.

[87]  David J. Zielinski,et al.  Medial prefrontal pathways for the contextual regulation of extinguished fear in humans , 2015, NeuroImage.

[88]  Russell A. Epstein,et al.  Anchoring the neural compass: Coding of local spatial reference frames in human medial parietal lobe , 2014, Nature Neuroscience.

[89]  Neil Burgess,et al.  Examining the role of the temporo-parietal network in memory, imagery, and viewpoint transformations , 2014, Front. Hum. Neurosci..

[90]  Inah Lee,et al.  Functional cross‐hemispheric shift between object‐place paired associate memory and spatial memory in the human hippocampus , 2016, Hippocampus.

[91]  P. G. Schrader,et al.  A sense of self: The role of presence in virtual environments , 2013, Comput. Hum. Behav..

[92]  Chris Dede,et al.  Immersive Interfaces for Engagement and Learning , 2009, Science.

[93]  Daniel Durstewitz,et al.  A statistical approach for segregating cognitive task stages from multivariate fMRI BOLD time series , 2015, Front. Hum. Neurosci..

[94]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[95]  G. Dawson,et al.  Scopolamine disrupts hippocampal activity during allocentric spatial memory in humans: an fMRI study using a virtual reality analogue of the Morris Water Maze , 2011, Journal of psychopharmacology.

[96]  M. Moser,et al.  Pattern Separation in the Dentate Gyrus and CA3 of the Hippocampus , 2007, Science.

[97]  Gabriele Janzen,et al.  Encoding and retrieval of landmark‐related spatial cues during navigation: An fMRI study , 2014, Hippocampus.

[98]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[99]  J. Caplan,et al.  Building a memory palace in minutes: equivalent memory performance using virtual versus conventional environments with the Method of Loci. , 2012, Acta psychologica.

[100]  Mayank R. Mehta,et al.  Multisensory Control of Hippocampal Spatiotemporal Selectivity , 2013, Science.

[101]  David J. Zielinski,et al.  Extinction in multiple virtual reality contexts diminishes fear reinstatement in humans , 2014, Neurobiology of Learning and Memory.

[102]  Floris P de Lange,et al.  How the human brain goes virtual: distinct cortical regions of the person-processing network are involved in self-identification with virtual agents. , 2012, Cerebral cortex.

[103]  Christian Büchel,et al.  Spatial updating: how the brain keeps track of changing object locations during observer motion , 2008, Nature Neuroscience.

[104]  David J. Zielinski,et al.  Revealing Context-Specific Conditioned Fear Memories with Full Immersion Virtual Reality , 2011, Front. Behav. Neurosci..

[105]  P. Pauli,et al.  Delay and trace fear conditioning in a complex virtual learning environment—neural substrates of extinction , 2014, Front. Hum. Neurosci..

[106]  Eleanor A. Maguire,et al.  Thoughts, behaviour, and brain dynamics during navigation in the real world , 2006, NeuroImage.

[107]  David C. Evans,et al.  Encoding and Retrieval , 2017 .

[108]  Jean Gotman,et al.  Low-frequency theta oscillations in the human hippocampus during real-world and virtual navigation , 2017, Nature Communications.

[109]  B. Christensen,et al.  Hippocampal activation and memory performance in schizophrenia depend on strategy use in a virtual maze , 2017, Psychiatry Research: Neuroimaging.

[110]  E. Spelke,et al.  Human Spatial Representation: Insights from Animals , 2002 .

[111]  V. Gyselinck,et al.  Using virtual reality to characterize episodic memory profiles in amnestic mild cognitive impairment and Alzheimer's disease: Influence of active and passive encoding , 2012, Neuropsychologia.

[112]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[113]  S. Tonegawa,et al.  Creating a False Memory in the Hippocampus , 2013, Science.

[114]  Mary Hegarty,et al.  The Human Retrosplenial Cortex and Thalamus Code Head Direction in a Global Reference Frame , 2016, The Journal of Neuroscience.

[115]  Elizabeth K. Baker,et al.  fMRI Hippocampal Activity During a VirtualRadial Arm Maze , 2005, Applied psychophysiology and biofeedback.

[116]  C. Büchel,et al.  Differential Recruitment of the Hippocampus, Medial Prefrontal Cortex, and the Human Motion Complex during Path Integration in Humans , 2007, The Journal of Neuroscience.

[117]  Jun Liu,et al.  A virtual reality-based FMRI study of reward-based spatial learning , 2010, Neuropsychologia.

[118]  Christian Büchel,et al.  Neural foundations of emerging route knowledge in complex spatial environments. , 2004, Brain research. Cognitive brain research.

[119]  P. Rodríguez Human navigation that requires calculating heading vectors recruits parietal cortex in a virtual and visually sparse water maze task in fMRI. , 2010, Behavioral neuroscience.