Learning real-life cognitive abilities in a novel 360°-virtual reality supermarket: a neuropsychological study of healthy participants and patients with epilepsy

BackgroundTo increase the ecological validity of neuropsychological instruments the use of virtual reality (VR) applications can be considered as an effective tool in the field of cognitive neurorehabilitation. Despite the growing use of VR programs, only few studies have considered the application of everyday activities like shopping or travelling in VR training devices.MethodsWe developed a novel 360°- VR supermarket, which is displayed on a circular arrangement of 8 touch-screens – the “OctaVis”. In this setting, healthy human adults had to memorize an auditorily presented shopping list (list A) and subsequently buy all remembered products of this list in the VR supermarket. This procedure was accomplished on three consecutive days. On day four, a new shopping list (list B) was introduced and participants had to memorize and buy only products of this list. On day five, participants had to buy all remembered items of list A again, but without new presentation of list A. Additionally, we obtained measures of participants’ presence, immersion and figural-spatial memory abilities. We also tested a sample of patients with focal epilepsy with an extended version of our shopping task, which consisted of eight days of training.ResultsWe observed a comprehensive and stable effect of learning for the number of correct products, the required time for shopping, and the length of movement trajectories in the VR supermarket in the course of the training program. Task performance was significantly correlated with participants’ figural-spatial memory abilities and subjective level of immersion into the VR.ConclusionsLearning effects in our paradigm extend beyond mere verbal learning of the shopping list as the data show evidence for multi-layered learning (at least visual-spatial, strategic, and verbal) on concordant measures. Importantly, learning also correlated with measures of figural-spatial memory and the degree of immersion into the VR. We propose that cognitive training with the VR supermarket program in the OctaVis will be efficient for the assessment and training of real-life cognitive abilities in healthy subjects and patients with epilepsy. It is most likely that our findings will also apply for patients with cognitive disabilities resulting from other neurological and psychiatric syndromes.

[1]  Frank Biocca,et al.  Visual Touch in Virtual Environments: An Exploratory Study of Presence, Multimodal Interfaces, and Cross-Modal Sensory Illusions , 2001, Presence: Teleoperators & Virtual Environments.

[2]  Lakhmi C. Jain,et al.  Advanced Computational Intelligence Paradigms in Healthcare 6. Virtual Reality in Psychotherapy, Rehabilitation, and Assessment , 2011 .

[3]  Mario Botsch,et al.  OctaVis: A Simple and Efficient Multi-View Rendering System , 2010 .

[4]  J. Szaflarski,et al.  The use of self-generation procedures facilitates verbal memory in individuals with seizure disorders , 2008, Epilepsy & Behavior.

[5]  S. Graham,et al.  Utilizing Virtual Reality to Improve the Ecological Validity of Clinical Neuropsychology: An fMRI Case Study Elucidating the Neural Basis of Planning by Comparing the Tower of London with a Three-Dimensional Navigation Task , 2009, Applied neuropsychology.

[6]  Albert A. Rizzo,et al.  Virtual Reality in Brain Damage Rehabilitation: Review , 2005, Cyberpsychology Behav. Soc. Netw..

[7]  Özgür A. Onur,et al.  Aging-related changes of neural mechanisms underlying visual-spatial working memory , 2012, Neurobiology of Aging.

[8]  Neff Walker,et al.  Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[9]  Jeonghun Ku,et al.  A Virtual Reality System for the Assessment and Rehabilitation of the Activities of Daily Living , 2003, Cyberpsychology Behav. Soc. Netw..

[10]  J. Kwon,et al.  Deficits of organizational strategy and visual memory in obsessive-compulsive disorder. , 2004, Neuropsychology.

[11]  P. Osterrieth,et al.  Le test de copie d'une figure complexe; contribution à l'étude de la perception et de la mémoire. , 1944 .

[12]  M. Marusan,et al.  Virtual Reality in Neurorehabilitation : Mental Rotation , 2007 .

[13]  R. Kizony,et al.  Performance within a virtual supermarket and its relationship to executive functions in post-stroke patients , 2006, 2006 International Workshop on Virtual Rehabilitation.

[14]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[15]  E. A. Attree,et al.  The specificity of memory enhancement during interaction with a virtual environment. , 1999, Memory.

[16]  Michael J. Singer,et al.  The Factor Structure of the Presence Questionnaire , 2005, Presence: Teleoperators & Virtual Environments.

[17]  Jun Soo Kwon,et al.  Clinical and empirical applications of the Rey–Osterrieth Complex Figure Test , 2006, Nature Protocols.

[18]  Patrick Péruch,et al.  Active versus passive learning and testing in a complex outside built environment , 2004, Cognitive Processing.

[19]  Silke Lux,et al.  Verbaler Lern- und Merkfähigkeitstest , 2001 .

[20]  Evelyne Klinger,et al.  Use of the Virtual Action Planning Supermarket for the Diagnosis of Mild Cognitive Impairment , 2009, Dementia and Geriatric Cognitive Disorders.

[21]  Joseph J. LaViola,et al.  A discussion of cybersickness in virtual environments , 2000, SGCH.

[22]  E. Caine.  Amnesic disorders. , 1993, The Journal of neuropsychiatry and clinical neurosciences.

[23]  Larry F. Hodges,et al.  Can Audio Enhance Visual Perception and Performance in a Virtual Environment? , 1999 .

[24]  H. Markowitsch,et al.  Recovery of mnestic functions after hypoxic brain damage , 1995 .

[25]  Richard E. Mayer,et al.  Multimedia aids to problem-solving transfer , 1999 .

[26]  Jean W. Wilkins,et al.  Organizational quality on the Rey-Osterrieth and Taylor Complex Figure Tests: A new scoring system. , 1993 .

[27]  P. Osterrieth.  Le test de copie d'une figure complexe , 1944 .

[28]  L. Binder.  Constructional strategies on complex figure drawings after unilateral brain damage. , 1982, Journal of clinical neuropsychology.

[29]  Daniel R. Montello,et al.  Spatial Information Theory A Theoretical Basis for GIS , 1995, Lecture Notes in Computer Science.

[30]  Heather R. Collins.  Pictorial encoding and testing impact recognition memory , 2007 .

[31]  K. Willmes,et al.  CVLT California Verbal Learning Test. German adaptation. Manual , 2008 .

[32]  Hunter G. Hoffman,et al.  Virtual Chess: Meaning Enhances Users' Sense of Presence in Virtual Environments , 1998, Int. J. Hum. Comput. Interact..

[33]  N. Pachana,et al.  Ecological validity in neuropsychological assessment: a case for greater consideration in research with neurologically intact populations. , 2006, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[34]  E. Tulving,et al.  Episodic and declarative memory: Role of the hippocampus , 1998, Hippocampus.

[35]  J. Olver,et al.  Evaluation of a community-based model of rehabilitation following traumatic brain injury , 2006, Neuropsychological rehabilitation.

[36]  L. Squire.  Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory , 1992, Journal of Cognitive Neuroscience.

[37]  Gereon R Fink,et al.  Differential remoteness and emotional tone modulate the neural correlates of autobiographical memory. , 2003, Brain : a journal of neurology.

[38]  Mel Slater,et al.  Using Presence Questionnaires in Reality , 2000, Presence: Teleoperators & Virtual Environments.

[39]  Barbara Tversky,et al.  Cognitive Maps, Cognitive Collages, and Spatial Mental Models , 1993, COSIT.

[40]  Thomas D. Parsons,et al.  Neuropsychological Assessment Using Virtual Environments: Enhanced Assessment Technology for Improved Ecological Validity , 2011 .

[41]  M. Mapstone.  Neuropsychological Assessment, 4th Edition , 2005, Neurology.

[42]  Albert A. Rizzo,et al.  Virtual Environment for Assessment of Neurocognitive Functioning: Virtual Reality Cognitive Performance Assessment Test , 2008, MMVR.

[43]  C. Elger,et al.  Enhancing memory for lists by grouped presentation and rehearsal: A pilot study in healthy subjects with unexpected results , 2009, Seizure.

[44]  Richard E. Mayer,et al.  Multimedia Learning: The Promise of Multimedia Learning , 2001 .

[45]  Evelyne Klinger,et al.  Performance within the virtual action planning supermarket (VAP-S): an executive function profile of three different populations suffering from deficits in the central nervous system , 2008 .

[46]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[47]  John Cromby,et al.  Successful transfer to the real world of skills practised in a virtual environment by students with severe learning difficulties , 1996 .

[48]  T. L. Bennett.  Neuropsychological evaluation in rehabilitation planning and evaluation of functional skills. , 2001, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[49]  Heidi Sveistrup,et al.  The Effect of Two Types of Virtual Reality on Voluntary Center of Pressure Displacement , 2003, Cyberpsychology Behav. Soc. Netw..

[50]  Gereon R. Fink,et al.  Recollections of one’s own past: the effects of aging and gender on the neural mechanisms of episodic autobiographical memory , 2005, Anatomy and Embryology.

[51]  Jonathan Freeman,et al.  A Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory , 2001, Presence: Teleoperators & Virtual Environments.

[52]  Albert A. Rizzo,et al.  Analysis of assets for virtual reality applications in neuropsychology , 2004 .

[53]  Gereon R. Fink,et al.  Modulatory Effects of Levodopa on Cognitive Control in Young but not in Older Subjects: A Pharmacological fMRI Study , 2011, Journal of Cognitive Neuroscience.

[54]  Patrice L Tamar Weiss,et al.  Training multitasking in a virtual supermarket: a novel intervention after stroke. , 2009, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[55]  Patrick Péruch,et al.  Acquisition of Spatial Knowledge Through Visual Exploration of Simulated Environments , 1995 .

[56]  Maureen Schmitter-Edgecombe,et al.  Improving the ecological validity of executive functioning assessment. , 2006, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[57]  Gianluca Castelnuovo,et al.  Virtual Reality based tools for the rehabilitation of cognitive and executive functions: the V-STORE , 2003, PsychNology J..

[58]  H. Markowitsch.  Functional Neuroanatomy of Learning and Memory , 2003 .

[59]  Matthew Lombard,et al.  The Role of Screen Size in Viewer Responses to Television Fare , 1997 .

[60]  Brenda K. Wiederhold,et al.  Lessons Learned From 600 Virtual Reality Sessions , 2000, Cyberpsychology Behav. Soc. Netw..

[61]  E. Tolman.  Cognitive maps in rats and men. , 1948, Psychological review.

[62]  Andrea Gaggioli,et al.  Presence and rehabilitation: toward second-generation virtual reality applications in neuropsychology , 2004, Journal of NeuroEngineering and Rehabilitation.

[63]  Christian E Elger,et al.  Chronic epilepsy and cognition , 2004, The Lancet Neurology.

[64]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[65]  Simona Luzzi,et al.  Non-verbal memory measured by Rey–Osterrieth Complex Figure B: normative data , 2011, Neurological Sciences.

[66]  Caroline Knight,et al.  Development of a simplified version of the multiple errands test for use in hospital settings , 2002 .

[67]  M. Schmitter-Edgecombe,et al.  The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills , 2003, Neuropsychology Review.

[68]  Evelyne Klinger,et al.  Virtual Action Planning in Parkinson's Disease: AControl Study , 2006, Cyberpsychology Behav. Soc. Netw..

[69]  Ton de Jong,et al.  Multimodal Versus Unimodal Instruction in a Complex Learning Context , 2002 .

[70]  M. Tarr,et al.  Virtual reality in behavioral neuroscience and beyond , 2002, Nature Neuroscience.

[71]  R. Heinrichs,et al.  Current and emergent applications of neuropsychological assessment: Problems of validity and utility. , 1990 .

[72]  Young Youn Kim,et al.  Effect of cognitive training focusing on organizational strategies in patients with obsessive‐compulsive disorder , 2006, Psychiatry and clinical neurosciences.

[73]  H. Markowitsch,et al.  Die Wirksamkeit von Gedächtnistrainings in der Behandlung von Gedächtnisstörungen , 1991 .

[74]  Patrice L. Weiss,et al.  Virtual reality rehabilitation for all: Vivid GX versus Sony PlayStation II EyeToy , 2004 .

[75]  Patrice L. Weiss,et al.  Textbook of Neural Repair and Rehabilitation: Virtual reality in neurorehabilitation , 2006 .

[76]  Edwin H. Blake,et al.  Cognitive maps in virtual environments: facilitation of learning through the use of innate spatial abilities , 2001, AFRIGRAPH '01.

[77]  F. Schmitt,et al.  Rey-Osterrieth complex figure: Psychometric characteristics in a geriatric sample , 1991 .