Dementia detection using automatic analysis of conversations

Abstract Neurogenerative disorders, like dementia, can affect a person’s speech, language and as a consequence, conversational interaction capabilities. A recent study, aimed at improving dementia detection accuracy, investigated the use of conversation analysis (CA) of interviews between patients and neurologists as a means to differentiate between patients with progressive neurodegenerative memory disorder (ND) and those with (non-progressive) functional memory disorders (FMD). However, doing manual CA is expensive and difficult to scale up for routine clinical use. In this paper, we present an automatic classification system using an intelligent virtual agent (IVA). In particular, using two parallel corpora of respectively neurologist- and IVA-led interactions, we show that using acoustic, lexical and CA-inspired features enable ND/FMD classification rates of 90.0% for the neurologist-patient conversations, and 90.9% for the IVA-patient conversations. Analysis of the differentiating potential of individual features show that some differences exist between the IVA and human-led conversations, for example in average turn length of patients.

[1]  Francois Bremond,et al.  A Virtual Agent for enhancing performance and engagement of older people with dementia in Serious Games , 2016 .

[2]  Steven Baker,et al.  Active ageing with avatars: a virtual exercise class for older adults , 2016, OZCHI.

[3]  Nicholas C. Firth,et al.  MODELLING EYE-TRACKING DATA TO DISCRIMINATE BETWEEN ALZHEIMER'S PATIENTS AND HEALTHY CONTROLS , 2017, Alzheimer's & Dementia.

[4]  N. Cercone,et al.  Automatic detection and rating of dementia of Alzheimer type through lexical analysis of spontaneous speech , 2005, IEEE International Conference Mechatronics and Automation, 2005.

[5]  Brian Roark,et al.  Spoken Language Derived Measures for Detecting Mild Cognitive Impairment , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[7]  Hans-Werner Gellersen,et al.  Monitoring dementia with automatic eye movements analysis , 2016 .

[8]  Daniel Povey,et al.  The Kaldi Speech Recognition Toolkit , 2011 .

[9]  Gábor Gosztolya,et al.  Automatic detection of mild cognitive impairment from spontaneous speech using ASR , 2015, INTERSPEECH.

[10]  Jesús B. Alonso,et al.  Feature selection for spontaneous speech analysis to aid in Alzheimer's disease diagnosis: A fractal dimension approach , 2015, Comput. Speech Lang..

[11]  P. Drew,et al.  Conversational assessment in memory clinic encounters: interactional profiling for differentiating dementia from functional memory disorders , 2016, Aging & mental health.

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Ewan Klein,et al.  Natural Language Processing with Python , 2009 .

[14]  Marcos Faúndez-Zanuy,et al.  On the Selection of Non-Invasive Methods Based on Speech Analysis Oriented to Automatic Alzheimer Disease Diagnosis , 2013, Sensors.

[15]  Sadaoki Furui,et al.  Differences between acoustic characteristics of spontaneous and read speech and their effects on speech recognition performance , 2008, Comput. Speech Lang..

[16]  Kathleen C. Fraser,et al.  Linguistic Features Identify Alzheimer's Disease in Narrative Speech. , 2015, Journal of Alzheimer's disease : JAD.

[17]  K. Forbes-McKay,et al.  Detecting subtle spontaneous language decline in early Alzheimer’s disease with a picture description task , 2005, Neurological Sciences.

[18]  Heidi Christensen,et al.  An Avatar-Based System for Identifying Individuals Likely to Develop Dementia , 2017, INTERSPEECH.

[19]  Robert J. Moore,et al.  Automated Transcription and Conversation Analysis , 2015 .

[20]  M. Trosset,et al.  Cross-Sectional Analysis of Alzheimer Disease Effects on Oral Discourse in a Picture Description Task , 1996, Alzheimer disease and associated disorders.

[21]  Hans Förstl,et al.  Suitability of the 6CIT as a screening test for dementia in primary care patients , 2013, Aging & mental health.

[22]  K. Bayles,et al.  Effects of working memory deficits on the communicative functioning of Alzheimer's dementia patients. , 2003, Journal of communication disorders.

[23]  Colleen Richey,et al.  Aided diagnosis of dementia type through computer-based analysis of spontaneous speech , 2014, CLPsych@ACL.

[24]  Mark Huckvale,et al.  Avatar therapy for persecutory auditory hallucinations: What is it and how does it work? , 2013, Psychosis.

[25]  Gene H. Lerner.  Conversation analysis. Studies from the first generation. , 2004 .

[26]  H. Christensen,et al.  Toward the Automation of Diagnostic Conversation Analysis in Patients with Memory Complaints. , 2017, Journal of Alzheimer's disease : JAD.

[27]  Albert A. Rizzo,et al.  Automatic Behavior Analysis During a Clinical Interview with a Virtual Human , 2016, MMVR.

[28]  B. Metternich,et al.  The syndrome of functional memory disorder: definition, etiology, and natural course. , 2008, The American journal of geriatric psychiatry : official journal of the American Association for Geriatric Psychiatry.

[29]  P. Drew,et al.  Towards diagnostic conversational profiles of patients presenting with dementia or functional memory disorders to memory clinics. , 2015, Patient education and counseling.

[30]  Gábor Gosztolya,et al.  Detecting Mild Cognitive Impairment from Spontaneous Speech by Correlation-Based Phonetic Feature Selection , 2016, INTERSPEECH.

[31]  Beatrice Santorini,et al.  The Penn Treebank: An Overview , 2003 .

[32]  N. Thomas,et al.  Relating Therapy for distressing auditory hallucinations: A pilot randomized controlled trial , 2017, Schizophrenia Research.

[33]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[34]  Peter Ladefoged,et al.  Phonation types: a cross-linguistic overview , 2001, J. Phonetics.

[35]  D. R. Boone,et al.  The potential of language tasks for identifying senile dementia. , 1982, The Journal of speech and hearing disorders.

[36]  Yogesan Kanagasingam,et al.  Innovative diagnostic tools for early detection of Alzheimer's disease , 2015, Alzheimer's & Dementia.

[37]  Frank Rudzicz,et al.  Using linguistic features longitudinally to predict clinical scores for Alzheimer’s disease and related dementias , 2015, SLPAT@Interspeech.

[38]  A Sutcliffe,et al.  Can you detect early dementia from an email? A proof of principle study of daily computer use to detect cognitive and functional decline , 2018, International journal of geriatric psychiatry.

[39]  Heidi Christensen,et al.  Diagnosing People with Dementia Using Automatic Conversation Analysis , 2016, INTERSPEECH.

[40]  Gorka Epelde,et al.  Natural Interaction between Avatars and Persons with Alzheimer's Disease , 2008, ICCHP.

[41]  Jack Sidnell,et al.  The handbook on conversation analysis , 2013 .

[42]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[43]  Mar Rus-Calafell,et al.  A virtual reality-integrated program for improving social skills in patients with schizophrenia: a pilot study. , 2014, Journal of behavior therapy and experimental psychiatry.

[44]  H. Hamilton.  Conversations with an Alzheimer's Patient: An Interactional Sociolinguistic Study , 1994 .

[45]  Elizabeth Shriberg,et al.  Spontaneous speech: how people really talk and why engineers should care , 2005, INTERSPEECH.

[46]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[47]  Yiannis Kompatsiaris,et al.  Evaluation of speech-based protocol for detection of early-stage dementia , 2013, INTERSPEECH.