A study into the automation of cognitive assessment tasks for delivery via the telephone: lessons for developing remote monitoring applications for the elderly.

BACKGROUND Cognitive assessments are valuable tools in assessing neurological conditions. They are critical in measuring deficits in cognitive function in an array of neurological disorders and during the ageing process. Automation of cognitive assessments is one way to address the increasing burden on medical resources for an ever increasing ageing population. OBJECTIVE This study investigated the suitability of using automated Interactive Voice Response (IVR) technology to deliver a suite of cognitive assessments to older adults using speech as the input modality. METHODS Several clinically valid and gold-standard cognitive assessments were selected for implementation in the IVR application. The IVR application was designed using human centred design principles to ensure the experience was as user friendly as possible. Sixty one participants completed two IVR assessments and one face to face (FF) assessment with a neuropsychologist. Completion rates for individual tests were inspected to identify those tests that are most suitable for administration via IVR technology. Interclass correlations were calculated to assess the reliability of the automated administration of the cognitive assessments across delivery modes. RESULTS While all participants successfully completed all automated assessments, variability in the completion rates for different cognitive tests was observed. Statistical analysis found significant interclass correlations for certain cognitive tests between the different modes of administration. Analysis also suggests that an initial FF assessment reduces the variability in cognitive test scores when introducing automation into such an assessment. CONCLUSION [corrected] This study has demonstrated the functional and cognitive reliability of administering specific cognitive tests using an automated, speech driven application. This study has defined the characteristics of existing cognitive tests that are suitable for such an automated delivery system and also informs on the limitations of other cognitive tests for this modality. This study presents recommendations for developing future large scale cognitive assessments.

[1]  Richard Goldstein,et al.  Test–re-test reliability of the Hopkins Verbal Learning Test-Revised in individuals with traumatic brain injury , 2012, Brain injury.

[2]  T. Tombaugh,et al.  Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming. , 1999, Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists.

[3]  L. Wahlund,et al.  Word sequence production in cognitive decline: Forward ever, backward never , 2008, Logopedics, phoniatrics, vocology.

[4]  J. Mundt,et al.  Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology , 2007, Journal of Neurolinguistics.

[5]  David A Bennett,et al.  Telephone assessment of cognitive function in the late-onset Alzheimer's disease family study. , 2010, Archives of neurology.

[6]  M. Folstein,et al.  Population-based norms for the Mini-Mental State Examination by age and educational level. , 1993, JAMA.

[7]  J. Mundt,et al.  Computer-automated dementia screening using a touch-tone telephone. , 2001, Archives of internal medicine.

[8]  I. Grant,et al.  Test-Retest Reliability of Component Process Variables Within the Hopkins Verbal Learning Test-Revised , 2005, Assessment.

[9]  Arne Jönsson,et al.  Wizard of Oz studies -- why and how , 1993, Knowl. Based Syst..

[10]  Ronald C. Petersen,et al.  Assessment of cognition in mild cognitive impairment: A comparative study , 2011, Alzheimer's & Dementia.

[11]  J. Brandt.  The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms. , 1991 .

[12]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[13]  Susan M Resnick,et al.  Validation of a Cognitive Assessment Battery Administered over the Telephone , 2012, Journal of the American Geriatrics Society.

[14]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[15]  Jennifer Balogh,et al.  Voice User Interface Design , 2004 .

[16]  A. Brickman,et al.  Telephone-based identification of mild cognitive impairment and dementia in a multicultural cohort. , 2011, Archives of neurology.

[17]  M. Marsiske,et al.  The Use of the Modified Telephone Interview for Cognitive Status (TICS-M) in the Detection of Amnestic Mild Cognitive Impairment , 2009, Journal of geriatric psychiatry and neurology.