Reliability and construct validity of a revised Baycrest Multiple Errands Test

ABSTRACT Surprisingly few ecologically-valid assessments of executive function exist, but the Baycrest Multiple Errands Test (BMET) shows promise in identifying executive impairment. The goal of the present study was to develop both a revised version of the assessment (BMET-R), to improve the test's ability to discriminate between patients and healthy participants, and an alternate form of the BMET-R to permit repeat testing. Sixteen individuals with acquired brain injury (ABI) due to stroke or trauma and 16 healthy participants completed a series of neuropsychological assessments, questionnaires, the BMET-R and its alternate form (in counterbalanced order). The results of the study indicated that participants with ABI omitted more tasks, broke more rules, and were less efficient than healthy participants on both the revised BMET-R and its alternate form. Moreover, significant correlations were found between the two versions of the BMET-R for task completions, omissions, errors, rule breaks and inefficiencies but few significant correlations were observed between the BMET-R versions and measures of executive dysfunction in everyday life. These results indicate that the two versions of the BMET-R are able to dissociate the performance of participants with ABI from that of healthy participants. However, despite overlaps in performance and correlations between the two versions of the BMET-R, they did not identically assess executive deficits. This suggests that caution should be used when constructing and validating alternate versions of performance-based assessments.

[1]  Kelly L. Hoover.  Divergent validity of the Wechsler Memory Scale-Fourth Edition (WMS-IV) and the Delis-Kaplan Executive Function System (D-KEFS) , 2018 .

[2]  N. Carlozzi.  Shipley Institute of Living Scale , 2017 .

[3]  H. Polatajko,et al.  Executive function, self-regulation and attribution in acquired brain injury: A scoping review , 2013, Neuropsychological rehabilitation.

[4]  N. Korner-Bitensky,et al.  Stroke-specific executive function assessment: a literature review of performance-based tools. , 2013, Australian occupational therapy journal.

[5]  K. Zakzanis,et al.  Ecologically Valid Assessment of Executive Dysfunction Using a Novel Virtual Reality Task in Patients with Acquired Brain Injury , 2012, Applied neuropsychology. Adult.

[6]  Giuseppe Riva,et al.  Validating the Neuro VR-Based Virtual Version of the Multiple Errands Test: Preliminary Results , 2012, PRESENCE: Teleoperators and Virtual Environments.

[7]  D. Tranel,et al.  Neuropsychological assessment, 5th ed. , 2012 .

[8]  I. Altman,et al.  Rasch measurement analysis of the Mayo-Portland Adaptability Inventory (MPAI-4) in a community-based rehabilitation sample. , 2011, Journal of neurotrauma.

[9]  N. Katz,et al.  Ecological Validity of the Multiple Errands Test (MET) on Discharge from Neurorehabilitation Hospital , 2011, OTJR : occupation, participation and health.

[10]  C. Rainville,et al.  The IADL Profile: Development, Content Validity, Intra- and Interrater Agreement , 2010, Canadian journal of occupational therapy. Revue canadienne d'ergotherapie.

[11]  Jonathan J. Evans,et al.  A real-life, ecologically valid test of executive functioning: The executive secretarial task , 2010, Journal of clinical and experimental neuropsychology.

[12]  Robert K. Heaton,et al.  Neuropsychology and the prediction of everyday functioning. , 2010 .

[13]  D. Stuss,et al.  Further development of the Multiple Errands Test: standardized scoring, reliability, and ecological validity for the Baycrest version. , 2009, Archives of physical medicine and rehabilitation.

[14]  J. Tsao,et al.  The rehabilitation of executive disorders: a guide to theory and practice, Michael Oddy, Andrew Worthington (Eds.). Oxford University Press (2009), ISBN: 978-0-19-856805-6; 363 pages. $74.50 , 2009 .

[15]  D. Stuss.  Rehabilitation of frontal lobe dysfunction: a working framework , 2008 .

[16]  T. Toulopoulou,et al.  Assessment of executive functions: review of instruments and identification of critical issues. , 2008, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[17]  D. Stuss,et al.  Cognitive Neurorehabilitation: Rehabilitation of frontal lobe functions , 2008 .

[18]  Peter J. Anderson,et al.  Towards a developmental model of executive function. , 2008 .

[19]  Peter J. Anderson,et al.  Executive functions and the frontal lobes: A lifespan perspective. , 2008 .

[20]  T. Wolf,et al.  Initial development of a work-related assessment of dysexecutive syndrome: the Complex Task Performance Assessment. , 2008, Work.

[21]  D. Stuss,et al.  Is there a dysexecutive syndrome? , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[22]  千敬 髙島,et al.  呼吸器疾患におけるAssessment of Motor and Process Skillsの応用 , 2007 .

[23]  D. Tranel,et al.  Impaired behavior on real-world tasks following damage to the ventromedial prefrontal cortex , 2007, Journal of clinical and experimental neuropsychology.

[24]  Helen M Hoenig,et al.  Executive function deficits in acute stroke. , 2007, Archives of physical medicine and rehabilitation.

[25]  R. Dean,et al.  TEST REVIEW: Dean C. Delis, Edith Kaplan & Joel H. Kramer, Delis Kaplan Executive Function System (D-KEFS), The Psychological Corporation, San Antonio, TX, 2001. $415.00 (complete kit) , 2006 .

[26]  John Whyte,et al.  Cognitive Rehabilitation Interventions for Executive Function: Moving from Bench to Bedside in Patients with Traumatic Brain Injury , 2006, Journal of Cognitive Neuroscience.

[27]  M. Hogge,et al.  Exploration of the neural substrates of executive functioning by functional neuroimaging , 2006, Neuroscience.

[28]  P. Burgess,et al.  The case for the development and use of “ecologically valid” measures of executive function in experimental and clinical neuropsychology , 2006, Journal of the International Neuropsychological Society.

[29]  T. Shallice,et al.  Multiple frontal systems controlling response speed , 2005, Neuropsychologia.

[30]  J. Cummings,et al.  The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool For Mild Cognitive Impairment , 2005, Journal of the American Geriatrics Society.

[31]  M. Schmitter-Edgecombe,et al.  The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills , 2003, Neuropsychology Review.

[32]  Caroline Knight,et al.  Ecological validity of a simplified version of the multiple errands shopping test. , 2003, Journal of the International Neuropsychological Society : JINS.

[33]  Myrna F. Schwartz,et al.  The Naturalistic Action Test: A standardised assessment for everyday action impairment , 2002 .

[34]  Caroline Knight,et al.  Development of a simplified version of the multiple errands test for use in hospital settings , 2002 .

[35]  D. Stuss,et al.  Adult clinical neuropsychology: lessons from studies of the frontal lobes. , 2002, Annual review of psychology.

[36]  G. Gioia,et al.  TEST REVIEW Behavior Rating Inventory of Executive Function , 2000 .

[37]  M. J. Emerson,et al.  The Unity and Diversity of Executive Functions and Their Contributions to Complex “Frontal Lobe” Tasks: A Latent Variable Analysis , 2000, Cognitive Psychology.

[38]  T. Shallice,et al.  The cognitive and neuroanatomical correlates of multitasking , 2000, Neuropsychologia.

[39]  P W Burgess,et al.  Strategy application disorder: the role of the frontal lobes in human multitasking , 2000, Psychological research.

[40]  B. Ober,et al.  Independent frontal-system deficits in schizophrenia: cognitive, clinical, and adaptive implications , 1999, Psychiatry Research.

[41]  John R. Crawford,et al.  Introduction to the Assessment of Attention and Executive Functioning , 1998 .

[42]  Jordan Grafman,et al.  The Effects of Frontal Lobe Damage on Everyday Problem Solving , 1996, Cortex.

[43]  A. Damasio.  REVIEW ■ : Toward a Neurobiology of Emotion and Feeling: Operational Concepts and Hypotheses , 1995 .

[44]  S. Sautter,et al.  Route‐finding: A measure of everyday executive functioning in the head‐injured adult , 1993 .

[45]  N. Varney,et al.  Psychosocial and executive deficits following closed head injury: Implications for orbital frontal cortex , 1993 .

[46]  Robert K. Heaton,et al.  Wisconsin Card Sorting Test Manual – Revised and Expanded , 1993 .

[47]  D. Royall,et al.  Bedside Assessment of Executive Cognitive Impairment: The Executive Interview , 1992, Journal of the American Geriatrics Society.

[48]  Donald T. Stuss,et al.  Frontal dysfunction' after traumatic brain injury , 1992 .

[49]  O. Selnes.  A Compendium of Neuropsychological Tests , 1991, Neurology.

[50]  T. Shallice,et al.  Deficits in strategy application following frontal lobe damage in man. , 1991, Brain : a journal of neurology.

[51]  J. Brandt.  The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms. , 1991 .

[52]  O. Spreen,et al.  A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary , 1991 .

[53]  H S Levin,et al.  Frontal Lobe Dysfunction following Closed Head Injury. A Review of the Literature , 1990, The Journal of nervous and mental disease.

[54]  A. Damasio,et al.  Severe disturbance of higher cognition after bilateral frontal lobe ablation: Patient EVR , 1986 .

[55]  L. Radloff.  The CES-D Scale , 1977 .

[56]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[57]  J. Pickova.  The Measurement and Appraisal of Adult Intelligence , 1959 .

[58]  D. Wechsler.  The measurement and appraisal of adult intelligence, 4th ed. , 1958 .