The Ecological and Construct Validity of a Newly Developed Measure of Executive Function: The Virtual Library Task

Abstract Virtual reality (VR) assessment paradigms have the potential to address the limited ecological validity of pen and paper measures of executive function (EF) and the pragmatic and reliability issues associated with functional measures. To investigate the ecological validity and construct validity of a newly developed VR measure of EF, the Virtual Library Task (VLT); a real life analogous task—the Real Library Task (RLT); and five neuropsychological measures of EF were administered to 30 patients with traumatic brain injury (TBI) and 30 healthy Controls. Significant others for each participant also completed the Dysexecutive Questionnaire (DEX), which is a behavioral rating scale of everyday EF. Performances on the VLT and the RLT were significantly positively correlated indicating that VR performance is similar to real world performance. The TBI group performed significantly worse than the Control group on the VLT and the Modified Six Elements Test (MSET) but the other four neuropsychological measures of EF failed to differentiate the groups. Both the MSET and the VLT significantly predicted everyday EF suggesting that they are both ecologically valid tools for the assessment of EF. The VLT has the advantage over the MSET of providing objective measurement of individual components of EF. (JINS, 2012, 18, 440–450)

[1]  Barry L. Richardson,et al.  Virtual Library Task , 2014 .

[2]  Bruce L. Miller,et al.  The Human Frontal Lobes: Functions and Disorders , 2013 .

[3]  R. Testa,et al.  Factor analysis of nineteen executive function tests in a healthy adult population. , 2012, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[4]  K. Greve,et al.  Executive dysfunction in traumatic brain injury: The effects of injury severity and effort on the Wisconsin Card Sorting Test , 2010, Journal of clinical and experimental neuropsychology.

[5]  Jonathan J. Evans,et al.  A real-life, ecologically valid test of executive functioning: The executive secretarial task , 2010, Journal of clinical and experimental neuropsychology.

[6]  D. Stuss,et al.  Further development of the Multiple Errands Test: standardized scoring, reliability, and ecological validity for the Baycrest version. , 2009, Archives of physical medicine and rehabilitation.

[7]  Noomi Katz,et al.  Validation of the Virtual MET as an assessment tool for executive functions , 2009, Neuropsychological rehabilitation.

[8]  N. Titov,et al.  Use of Virtual Reality Tasks to Assess Prospective Memory: Applicability and Evidence , 2009, Brain Impairment.

[9]  C. Silver,et al.  Relationship Between Parenting Stress and Ratings of Executive Functioning in Children With ADHD , 2009 .

[10]  Charles J Duffy,et al.  Detecting navigational deficits in cognitive aging and Alzheimer disease using virtual reality , 2008, Neurology.

[11]  J. Ponsford,et al.  Cognitive functioning ten years following traumatic brain injury and rehabilitation. , 2008, Neuropsychology.

[12]  T. Toulopoulou,et al.  Assessment of executive functions: review of instruments and identification of critical issues. , 2008, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[13]  A. Mortimer,et al.  Attention and executive function in people with schizophrenia: Relationship with social skills and quality of life , 2008, International journal of psychiatry in clinical practice.

[14]  N. Katz,et al.  Validity of the Executive Function Performance Test in Individuals with Schizophrenia , 2007 .

[15]  Evelyne Klinger,et al.  Virtual Action Planning in Parkinson's Disease: AControl Study , 2006, Cyberpsychology Behav. Soc. Netw..

[16]  N. Pachana,et al.  Ecological validity in neuropsychological assessment: a case for greater consideration in research with neurologically intact populations. , 2006, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[17]  Maureen Schmitter-Edgecombe,et al.  Improving the ecological validity of executive functioning assessment. , 2006, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[18]  P. Burgess,et al.  The case for the development and use of “ecologically valid” measures of executive function in experimental and clinical neuropsychology , 2006, Journal of the International Neuropsychological Society.

[19]  J. Duncan.  Frontal Lobe Function and General Intelligence: Why it Matters , 2005, Cortex.

[20]  R. Vanderploeg,et al.  The Components of Executive Functioning in Traumatic Brain Injury , 2005, Journal of clinical and experimental neuropsychology.

[21]  J. Ponsford,et al.  Assessment of executive dysfunction following traumatic brain injury: Comparison of the BADS with other clinical neuropsychological measures , 2005, Journal of the International Neuropsychological Society.

[22]  J. Ponsford,et al.  Measuring executive dysfunction in an acute rehabilitation setting: Using the dysexecutive questionnaire (DEX) , 2005, Journal of the International Neuropsychological Society.

[23]  R. Jung.  Neuropsychological Assessment, 4th ed. , 2005 .

[24]  N. Titov,et al.  A computer-based procedure for assessing functional cognitive skills in patients with neurological injuries: The virtual street , 2005, Brain injury.

[25]  H. Jackson,et al.  The assessment of executive functions: coming out of the office , 2004, Brain injury.

[26]  G. Riva.  Cybertherapy - Internet and Virtual Reality as Assessment and Rehabilitation Tools for Clinical Psychology and Neuroscience , 2004, Cybertherapy.

[27]  R. Mccaffrey,et al.  The Practice of Forensic Neuropsychology: Meeting Challenges in the Courtroom , 2004 .

[28]  F. D. Rose,et al.  Assessing stroke patients’ prospective memory using virtual reality , 2004, Brain injury.

[29]  M. Schmitter-Edgecombe,et al.  The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills , 2003, Neuropsychology Review.

[30]  Kenneth J Ottenbacher,et al.  A virtual reality environment for evaluation of a daily living skill in brain injury rehabilitation: reliability and validity. , 2003, Archives of physical medicine and rehabilitation.

[31]  I. Robertson,et al.  Rehabilitation of executive function: facilitation of effective goal management on complex tasks using periodic auditory alerts , 2002, Neuropsychologia.

[32]  R. Chan,et al.  The application of “dysexecutive syndrome” measures across cultures: Performance and checklist assessment in neurologically healthy and traumatically brain-injured Hong Kong Chinese volunteers , 2002, Journal of the International Neuropsychological Society.

[33]  A. Yoshino,et al.  Executive dysfunction and clinical outcome in chronic alcoholics. , 2002, Alcoholism, clinical and experimental research.

[34]  Caroline Knight,et al.  Development of a simplified version of the multiple errands test for use in hospital settings , 2002 .

[35]  Peter McGeorge,et al.  Using Virtual Environments in the Assessment of Executive Dysfunction , 2001, Presence: Teleoperators & Virtual Environments.

[36]  James S. Elkind,et al.  A Simulated Reality Scenario Compared with the Computerized Wisconsin Card Sorting Test: An Analysis of Preliminary Results , 2001, Cyberpsychology Behav. Soc. Netw..

[37]  R. Chan,et al.  Dysexecutive symptoms among a non-clinical sample: a study with the use of the Dysexecutive Questionnaire. , 2001, British journal of psychology.

[38]  R. Tate,et al.  The Behavioural Assessment of the Dysexecutive Syndrome (BADS): Ecological, Concurrent and Construct Validity , 2000 .

[39]  P. Burgess,et al.  The ecological validity of tests of executive function , 1998, Journal of the International Neuropsychological Society.

[40]  Paul W. Burgess,et al.  The Development of an Ecologically Valid Test for Assessing Patients with a Dysexecutive Syndrome , 1998 .

[41]  Paul W. Burgess,et al.  Behavioural Assessment of the Dysexecutive Syndrome , 1997 .

[42]  Michaela Swales,et al.  Frontal Lobe Deficits after Head Injury: Unity and Diversity of Function , 1997 .

[43]  M. Williams,et al.  Factorial structure of the Wisconsin Card Sorting Test. , 1997, The British journal of clinical psychology.

[44]  P. McKenna,et al.  Assessment of the dysexecutive syndrome in schizophrenia , 1997, Psychological Medicine.

[45]  L L Thompson,et al.  Discriminability of the Wisconsin Card Sorting Test using the standardization sample. , 1996, Journal of clinical and experimental neuropsychology.

[46]  J. Fuster.  Frontal lobes , 1993, Current Opinion in Neurobiology.

[47]  J. Ponsford,et al.  The use of a rating scale of attentional behaviour , 1991 .

[48]  T. Shallice,et al.  Deficits in strategy application following frontal lobe damage in man. , 1991, Brain : a journal of neurology.

[49]  J. Batchelor,et al.  Preliminary validation of a clinical scale for measuring the duration of post‐traumatic amnesia , 1986, The Medical journal of Australia.

[50]  D. Stuss,et al.  The Frontal Lobes , 1986 .

[51]  Arthur L. Benton,et al.  Differential behavioral effects in frontal lobe disease , 1968 .

[52]  J. Henry,et al.  Assessment of executive dysfunction , 2012 .

[53]  R. Benoit,et al.  Mesulam's frontal lobe mystery re-examined. , 2009, Restorative neurology and neuroscience.

[54]  B. Miller,et al.  The human frontal lobes: Functions and disorders, 2nd ed. , 2007 .

[55]  Donald T. Stuss,et al.  NEW APPROACHES TO PREFRONTAL LOBE TESTING , 2006 .

[56]  Caroline Knight,et al.  Ecological validity of a simplified version of the multiple errands shopping test. , 2003, Journal of the International Neuropsychological Society : JINS.

[57]  Francis Rose,et al.  Virtual reality investigation of strategy formation, rule breaking and prospective memory in patients with focal prefrontal neurosurgical lesions , 2002 .

[58]  F. Morganti,et al.  Internet and Virtual Reality as Assessment and Rehabilitation Tools for Clinical Psychology and Neuroscience 4 Virtual Interaction in Cognitive Neuropsychology , 2022 .