Trends on the application of serious games to neuropsychological evaluation: A scoping review

BACKGROUND The dramatic technological advances witnessed in recent years have resulted in a great opportunity for changing the way neuropsychological evaluations may be performed in clinical practice. Particularly, serious games have been posed as the cornerstone of this still incipient paradigm-shift, as they have characteristics that make them especially advantageous in trying to overcome limitations associated with traditional pen-and-paper based neuropsychological tests: they can be easily administered and they can feature complex environments for the evaluation of neuropsychological constructs that are difficult to evaluate through traditional tests. The objective of this study was to conduct a scoping literature review in order to map rapidly the key concepts underpinning this research area during the last 25years on the use of serious games for neuropsychological evaluation. METHODS MEDLINE, PsycINFO, Scopus and IEEE Xplore databases were systematically searched. The main eligibility criteria were to select studies published in a peer-reviewed journal; written in English; published in the last 25years; focused on the human population, and classified in the neuropsychological field. Moreover, to avoid risk of bias, studies were selected by consensus of experts, focusing primarily in psychometric properties. Therefore, selected studies were analyzed in accordance with a set of dimensions of analysis commonly used for evaluating neuropsychological tests. RESULTS After applying the selected search strategy, 57 studies -including 54 serious games- met our selection criteria. The selected studies deal with visuospatial capabilities, memory, attention, executive functions, and complex neuropsychological constructs such as Mild Cognitive Impairment (MCI). Results show that the implementation of serious games for neuropsychological evaluation is tackled in several different ways in the selected studies, and that studies have so far been mainly exploratory, just aiming at testing the feasibility of the proposed approaches. DISCUSSION It may be argued that the limited number of databases used might compromise this study. However, we think that the finally included sample is representative, in spite of how difficult is to achieve an optimum and maximum scope. Indeed, this review identifies other research issues related to the development of serious games beyond their reliability and validity. The main conclusion of this review is that there is a great interest in the research community in the use of serious games for neuropsychological evaluation. This scoping review is pertinent, in accordance with the increasing number of studies published in the last three years, they demonstrate its potential as a serious alternative to classic neuropsychological tests. Nevertheless, more research is needed in order to implement serious games that are reliable, valid, and ready to be used in the everyday clinical practice.

[1]  N. Titov,et al.  Use of Virtual Reality Tasks to Assess Prospective Memory: Applicability and Evidence , 2009, Brain Impairment.

[2]  Dimitrios Tzovaras,et al.  Can a virtual reality cognitive training application fulfill a dual role? Using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. , 2015, Journal of Alzheimer's disease : JAD.

[3]  O. Spreen,et al.  A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary , 1991 .

[4]  M. McDaniel,et al.  Strategic and automatic processes in prospective memory retrieval: a multiprocess framework , 2000 .

[5]  B. Richardson,et al.  The Ecological and Construct Validity of a Newly Developed Measure of Executive Function: The Virtual Library Task , 2012, Journal of the International Neuropsychological Society.

[6]  Sergi Grau,et al.  Serious games for screening pre-dementia conditions: from virtuality to reality? A pilot project. , 2014, Functional neurology.

[7]  J. Teresi,et al.  The Free and Cued Selective Reminding Test: evidence of psychometric adequacy , 2009 .

[8]  Torsten Kuhlen,et al.  Evaluation of Spatial Processing in Virtual Reality Using Functional Magnetic Resonance Imaging (fMRI) , 2009, Cyberpsychology Behav. Soc. Netw..

[9]  Sarah Tomaszewski Farias,et al.  The relationship between neuropsychological performance and daily functioning in individuals with Alzheimer's disease: ecological validity of neuropsychological tests. , 2003, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[10]  Myrna F. Schwartz,et al.  The Naturalistic Action Test: A standardised assessment for everyday action impairment , 2002 .

[11]  B. Lawlor,et al.  Detection of prospective memory deficits in mild cognitive impairment of suspected Alzheimer’s disease etiology using a novel event-based prospective memory task , 2009, Journal of the International Neuropsychological Society.

[12]  Ioannis Tarnanas,et al.  Ecological Validity of Virtual Reality Daily Living Activities Screening for Early Dementia: Longitudinal Study , 2013, JMIR serious games.

[13]  U. Díaz-Orueta,et al.  AULA: An ecological virtual reality test with distractors for evaluating attention in children and adolescents , 2012 .

[14]  Valeria Manera,et al.  ‘Kitchen and cooking,’ a serious game for mild cognitive impairment and Alzheimer’s disease: a pilot study , 2015, Front. Aging Neurosci..

[15]  J. L. Ellis,et al.  Varieties of intention: Some distinctions and classifications. , 1996 .

[16]  D. Salmon,et al.  The neuropsychological profile of Alzheimer disease. , 2012, Cold Spring Harbor perspectives in medicine.

[17]  L. Cronbach Coefficient alpha and the internal structure of tests , 1951 .

[18]  E. A. Attree,et al.  An Assessment of Prospective Memory Retrieval in Women with Chronic Fatigue Syndrome Using a Virtual-Reality Environment: An Initial Study , 2009, Cyberpsychology Behav. Soc. Netw..

[19]  Z. Janka,et al.  Disturbed visuo-spatial orientation in the early stage of Alzheimer's dementia. , 1995, Archives of gerontology and geriatrics.

[20]  Thomas D. Parsons,et al.  Neuropsychological Assessment of Attentional Processing using Virtual Reality , 2008 .

[21]  H. Buschke,et al.  Genuine memory deficits in dementia , 1987 .

[22]  Albert A. Rizzo,et al.  The Virtual Classroom: A Virtual Reality Environment for the Assessment and Rehabilitation of Attention Deficits , 2000, Cyberpsychology Behav. Soc. Netw..

[23]  A. Baddeley,et al.  The development and validation of a test battery for detecting and monitoring everyday memory problems. , 1989, Journal of clinical and experimental neuropsychology.

[24]  Gábor Csukly,et al.  Monitoring the Early Signs of Cognitive Decline in Elderly by Computer Games: An MRI Study , 2015, PloS one.

[25]  Minh Quach,et al.  Screening for mild cognitive impairment (MCI) utilizing combined mini‐mental‐cognitive capacity examinations for identifying dementia prodromes , 2002, International journal of geriatric psychiatry.

[26]  Thomas D. Parsons,et al.  Virtual Teacher and Classroom for Assessment of Neurodevelopmental Disorders , 2014, Technologies of Inclusive Well-Being.

[27]  Evelyne Klinger,et al.  Use of the Virtual Action Planning Supermarket for the Diagnosis of Mild Cognitive Impairment , 2009, Dementia and Geriatric Cognitive Disorders.

[28]  B. Pennington,et al.  A comparison of the cognitive deficits in reading disability and attention-deficit/hyperactivity disorder. , 2001, Journal of abnormal psychology.

[29]  P. Piolino,et al.  The influence of action on episodic memory: A virtual reality study , 2013, Quarterly journal of experimental psychology.

[30]  B. N’Kaoua,et al.  [Virtual reality for clinical assessment of elderly people: early screening for dementia]. , 2012, Revue neurologique.

[31]  J. Galen Buckwalter,et al.  Sex differences in mental rotation and spatial rotation in a virtual environment , 2004, Neuropsychologia.

[32]  Déborah Alexandra Foloppe,et al.  Detecting Everyday Action Deficits in Alzheimer’s Disease Using a Nonimmersive Virtual Reality Kitchen , 2014, Journal of the International Neuropsychological Society.

[33]  Maureen K. Holden,et al.  Virtual Environments for Motor Rehabilitation: Review , 2005, Cyberpsychology Behav. Soc. Netw..

[34]  Unai Díaz-Orueta,et al.  AULA virtual reality test as an attention measure: Convergent validity with Conners’ Continuous Performance Test , 2014, Child neuropsychology : a journal on normal and abnormal development in childhood and adolescence.

[35]  Raja Parasuraman,et al.  Can we create a measure of fluid intelligence using Puzzle Creator within Portal 2 , 2016 .

[36]  A. Aron The Neural Basis of Inhibition in Cognitive Control , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[37]  Misha Pavel,et al.  Assessing Executive Function Using a Computer Game: Computational Modeling of Cognitive Processes , 2014, IEEE Journal of Biomedical and Health Informatics.

[38]  Tom Manly,et al.  The sensitivity of a virtual reality task to planning and prospective memory impairments: Group differences and the efficacy of periodic alerts on performance , 2010, Neuropsychological rehabilitation.

[39]  K. Abe,et al.  Computerized touch-panel screening tests for detecting mild cognitive impairment and Alzheimer's disease. , 2015, Internal medicine.

[40]  M. W. Richardson,et al.  The theory of the estimation of test reliability , 1937 .

[41]  Giuseppe Riva,et al.  Validation of a Neuro Virtual Reality-based Version of the Multiple Errands Test for the Assessment of Executive Functions , 2011, Annual Review of Cybertherapy and Telemedicine.

[42]  Philippe Allain,et al.  ClinicaVR: Classroom-CPT: A virtual reality tool for assessing attention and inhibition in children and adolescents , 2016, Comput. Hum. Behav..

[43]  J. Galen Buckwalter,et al.  A Controlled Clinical Comparison of Attention Performance in Children with ADHD in a Virtual Reality Classroom Compared to Standard Neuropsychological Methods , 2007, Child neuropsychology : a journal on normal and abnormal development in childhood and adolescence.

[44]  B. N'Kaoua,et al.  Age and active navigation effects on episodic memory: A virtual reality study. , 2016, British journal of psychology.

[45]  Maarten Löffler,et al.  Ecological validity of virtual environments to assess human navigation ability , 2015, Front. Psychol..

[46]  S. Vandenberg,et al.  Mental Rotations, a Group Test of Three-Dimensional Spatial Visualization , 1978, Perceptual and motor skills.

[47]  E. Tulving Episodic memory: from mind to brain. , 2002, Annual review of psychology.

[48]  David H K Shum,et al.  Evaluation of a virtual reality prospective memory task for use with individuals with severe traumatic brain injury , 2014, Neuropsychological rehabilitation.

[49]  Jonathan J. Evans,et al.  Prospective memory functioning in people with and without brain injury , 2002, Journal of the International Neuropsychological Society.

[50]  Sergi Grau,et al.  A Serious Games platform for early diagnosis of mild cognitive impairments , 2014, GFHEU.

[51]  Oonagh Thompson,et al.  Examining the Neurocognitive Validity of Commercially Available, Smartphone-Based Puzzle Games , 2012 .

[52]  Ioannis Tarnanas,et al.  Reliability of a novel serious game using dual-task gait profiles to early characterize aMCI , 2015, Front. Aging Neurosci..

[53]  Antonella Guidazzoli,et al.  The virtual reality Walking Corsi Test , 2015, Comput. Hum. Behav..

[54]  Daniel S Pine,et al.  Becoming the center of attention in social anxiety disorder: startle reactivity to a virtual audience during speech anticipation. , 2011, The Journal of clinical psychiatry.

[55]  Ricardo Rosas Díaz,et al.  TENI: A comprehensive battery for cognitive assessment based on games and technology , 2016, Child neuropsychology : a journal on normal and abnormal development in childhood and adolescence.

[56]  Mark H. Chignell,et al.  Developing a serious game for cognitive assessment: choosing settings and measuring performance , 2014, Chinese CHI '14.

[57]  V. Gyselinck,et al.  Using virtual reality to characterize episodic memory profiles in amnestic mild cognitive impairment and Alzheimer's disease: Influence of active and passive encoding , 2012, Neuropsychologia.

[58]  P. Nolin,et al.  Development and initial assessment of a new paradigm for assessing cognitive and motor inhibition: The bimodal virtual-reality Stroop , 2012, Journal of Neuroscience Methods.

[59]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[60]  K. Abe,et al.  Simultaneous assessment of cognitive and affective functions in multiple system atrophy and cortical cerebellar atrophy in relation to computerized touch-panel screening tests , 2015, Journal of the Neurological Sciences.

[61]  E. Tulving,et al.  Memory Systems 1994 , 1994 .

[62]  Caroline Knight,et al.  Ecological validity of a simplified version of the multiple errands shopping test. , 2003, Journal of the International Neuropsychological Society : JINS.

[63]  É. Sorita,et al.  Everyday-like memory and its cognitive correlates in healthy older adults and in young patients with traumatic brain injury: a pilot study based on virtual reality , 2014, Disability and rehabilitation. Assistive technology.

[64]  U. Díaz-Orueta,et al.  AULA—Advanced Virtual Reality Tool for the Assessment of Attention , 2016, Journal of attention disorders.

[65]  David R. Michael,et al.  Serious Games: Games That Educate, Train, and Inform , 2005 .

[66]  M. McDaniel,et al.  Normal aging and prospective memory. , 1990, Journal of experimental psychology. Learning, memory, and cognition.

[67]  G. Fisone,et al.  Acquisition and expression of conditioned taste aversion differentially affects extracellular signal regulated kinase and glutamate receptor phosphorylation in rat prefrontal cortex and nucleus accumbens , 2014, Front. Behav. Neurosci..

[68]  Giuseppe Riva,et al.  Detecting early egocentric and allocentric impairments deficits in Alzheimer’s disease: an experimental study with virtual reality , 2015, Front. Aging Neurosci..

[69]  Anna S. Law,et al.  Event-Based Prospective Remembering in a Virtual World , 2011, Quarterly journal of experimental psychology.

[70]  Til Wykes,et al.  Virtual shopping: A viable alternative to direct assessment of real life function? , 2016, Schizophrenia Research.

[71]  Evelyn H. Kroesbergen,et al.  Validity and reliability of an online visual–spatial working memory task for self-reliant administration in school-aged children , 2014, Behavior Research Methods.

[72]  M. McDaniel,et al.  Prospective Memory: An Overview and Synthesis of an Emerging Field , 2007 .

[73]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[74]  L M Greenberg,et al.  Developmental normative data on the test of variables of attention (T.O.V.A.). , 1993, Journal of child psychology and psychiatry, and allied disciplines.

[75]  Michael F. Bunting,et al.  Measuring working memory is all fun and games: a four-dimensional spatial game predicts cognitive task performance. , 2014, Experimental psychology.

[76]  J. Lani,et al.  Conners' Continuous Performance Test II (CPT II) , 2002 .

[77]  Gayle Brewer,et al.  Performance on the traditional and the touch screen, tablet versions of the Corsi Block and the Tower of Hanoi tasks , 2016, Comput. Hum. Behav..

[78]  R. Salvador,et al.  Examining hippocampal function in schizophrenia using a virtual reality spatial navigation task , 2016, Schizophrenia Research.

[79]  Albert A Rizzo,et al.  The Utility of a Continuous Performance Test Embedded in Virtual Reality in Measuring ADHD-Related Deficits , 2009, Journal of developmental and behavioral pediatrics : JDBP.

[80]  Ioannis Tarnanas,et al.  Can a novel computerized cognitive screening test provide additional information for early detection of Alzheimer's disease? , 2014, Alzheimer's & Dementia.

[81]  Christopher G. Courtney,et al.  Validity of the Virtual Reality Stroop Task (VRST) in active duty military , 2013, Journal of clinical and experimental neuropsychology.

[82]  R. Saan,et al.  Behavioural assessment of the dysexecutive syndrome (BADS) , 2012 .

[83]  Xia Zheng,et al.  The use of virtual reality for episodic memory assessment: effects of active navigation. , 2011, Experimental psychology.

[84]  Kiwan Han,et al.  Assessment of cognitive flexibility in real life using virtual reality: A comparison of healthy individuals and schizophrenia patients , 2012, Comput. Biol. Medicine.

[85]  Ashok S. Jansari,et al.  Ecological Assessment of Executive Functions: A New Virtual Reality Paradigm , 2014, Brain Impairment.

[86]  J. Henry,et al.  Prospective memory in multiple sclerosis , 2007, Journal of the International Neuropsychological Society.

[87]  Roger Smith,et al.  The Long History of Gaming in Military Training , 2010 .

[88]  Evelyne Klinger,et al.  Using the virtual action planning-supermarket for evaluating executive functions in people with stroke. , 2014, Journal of stroke and cerebrovascular diseases : the official journal of National Stroke Association.

[89]  D. Moher,et al.  Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement , 2009, BMJ : British Medical Journal.

[90]  M. Schmitter-Edgecombe,et al.  The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills , 2003, Neuropsychology Review.

[91]  J. Henry,et al.  A Virtual Week study of prospective memory function in autism spectrum disorders. , 2014, Journal of experimental child psychology.

[92]  T. Hashimoto,et al.  Event based and time based prospective memory in Parkinson’s disease , 2003, Journal of neurology, neurosurgery, and psychiatry.

[93]  Roy P C Kessels,et al.  Puzzling With Online Games (BAM-COG): Reliability, Validity, and Feasibility of an Online Self-Monitor for Cognitive Performance in Aging Adults , 2013, Journal of medical Internet research.

[94]  Philip M. Corsi Human memory and the medial temporal region of the brain. , 1972 .

[95]  J. Stroop Studies of interference in serial verbal reactions. , 1992 .

[96]  P. Philip,et al.  Cognitive evaluation by tasks in a virtual reality environment in multiple sclerosis , 2015, Journal of the Neurological Sciences.

[97]  Pierre Nolin,et al.  Multitasking and Prospective Memory: Can Virtual Reality be Useful for Diagnosis? , 2011, Behavioural neurology.

[98]  Malaz Boustani,et al.  Alzheimer's Association recommendations for operationalizing the detection of cognitive impairment during the Medicare Annual Wellness Visit in a primary care setting , 2013, Alzheimer's & Dementia.

[99]  Sara Rosenblum,et al.  Using a Virtual Classroom environment to describe the attention deficits profile of children with Neurofibromatosis type 1. , 2011, Research in developmental disabilities.

[100]  Exogenous orienting of attention in hearing: a virtual reality paradigm to assess auditory attention in neglect patients , 2016, Experimental Brain Research.

[101]  D. Delis,et al.  Profiles of demented and amnesic patients on the California verbal learning test : implications for the assessment of memory disorders , 1991 .

[102]  Giuseppe Riva,et al.  Virtual multiple errands test (VMET): a virtual reality-based tool to detect early executive functions deficit in Parkinson’s disease , 2014, Front. Behav. Neurosci..

[103]  Frederick A. Schmitt,et al.  Should older adults be screened for dementia? It is important to screen for evidence of dementia! , 2007, Alzheimer's & Dementia.

[104]  A. Postma,et al.  Remember to Buy Milk on the Way Home! A Meta-analytic Review of Prospective Memory in Mild Cognitive Impairment and Dementia , 2012, Journal of the International Neuropsychological Society.

[105]  Alain Berthoz,et al.  Effects of Enactment in Episodic Memory: A Pilot Virtual Reality Study with Young and Elderly Adults , 2014, Front. Aging Neurosci..

[106]  Frédéric Banville,et al.  Virtual Reality as a New Approach to Assess Cognitive Decline in the Elderly , 2013 .

[107]  Pascal Guitton,et al.  Everyday-like memory for objects in ageing and Alzheimer's disease assessed in a visually complex environment: The role of executive functioning and episodic memory. , 2016, Journal of neuropsychology.

[108]  Jung-Seok Choi,et al.  Spatial memory impairments in amnestic mild cognitive impairment in a virtual radial arm maze , 2014, Neuropsychiatric disease and treatment.