An Assessment of Paralinguistic Acoustic Features for Detection of Alzheimer's Dementia in Spontaneous Speech

Speech analysis could provide an indicator of Alzheimer's disease and help develop clinical tools for automatically detecting and monitoring disease progression. While previous studies have employed acoustic (speech) features for characterisation of Alzheimer's dementia, these studies focused on a few common prosodic features, often in combination with lexical and syntactic features which require transcription. We present a detailed study of the predictive value of purely acoustic features automatically extracted from spontaneous speech for Alzheimer's dementia detection, from a computational paralinguistics perspective. The effectiveness of several state-of-the-art paralinguistic feature sets for Alzheimer's detection were assessed on a balanced sample of DementiaBank's Pitt spontaneous speech dataset, with patients matched by gender and age. The feature sets assessed were the extended Geneva minimalistic acoustic parameter set (eGeMAPS), the emobase feature set, the ComParE 2013 feature set, and new Multi-Resolution Cochleagram (MRCG) features. Furthermore, we introduce a new active data representation (ADR) method for feature extraction in Alzheimer's dementia recognition. Results show that classification models based solely on acoustic speech features extracted through our ADR method can achieve accuracy levels comparable to those achieved by models that employ higher-level language features. Analysis of the results suggests that all feature sets contribute information not captured by other feature sets. We show that while the eGeMAPS feature set provides slightly better accuracy than other feature sets individually (71.34%), “hard fusion” of feature sets improves accuracy to 78.70%.

[1]  Doris Ma Fat,et al.  WHO Library Cataloguing-in-Publication Data , 2001 .

[2]  Saturnino Luz,et al.  Attitude Recognition Using Multi-resolution Cochleagram Features , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Markus Reuber,et al.  Can conversation analytic findings help with differential diagnosis in routine seizure clinic interactions? , 2016, Communication & medicine.

[4]  Gábor Gosztolya,et al.  A Speech Recognition-based Solution for the Automatic Detection of Mild Cognitive Impairment from Spontaneous Speech , 2018, Current Alzheimer research.

[5]  Minsoo Hahn,et al.  Voice Activity Detection Using an Adaptive Context Attention Model , 2018, IEEE Signal Processing Letters.

[6]  Steven H Ferris,et al.  Language impairment in Alzheimer’s disease and benefits of acetylcholinesterase inhibitors , 2013, Clinical interventions in aging.

[7]  Sylvester Olubolu Orimaye,et al.  Deep language space neural network for classifying mild cognitive impairment and Alzheimer-type dementia , 2018, PloS one.

[8]  Saturnino Luz,et al.  Automatic identification of experts and performance prediction in the multimodal math data corpus through analysis of speech interaction , 2013, ICMI '13.

[9]  J. Hardy,et al.  The Amyloid Hypothesis of Alzheimer ’ s Disease : Progress and Problems on the Road to Therapeutics , 2009 .

[10]  Robert P. Friedland,et al.  Dementia: A Clinical Approach (2nd Ed.) , 1992 .

[11]  Maureen Schmitter-Edgecombe,et al.  Automated Cognitive Health Assessment Using Smart Home Monitoring of Complex Tasks , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[12]  Yiannis Kompatsiaris,et al.  Evaluation of speech-based protocol for detection of early-stage dementia , 2013, INTERSPEECH.

[13]  Michael S. Rafii,et al.  The natural history of Alzheimer’s disease , 2017 .

[14]  K. Forbes-McKay,et al.  Detecting subtle spontaneous language decline in early Alzheimer’s disease with a picture description task , 2005, Neurological Sciences.

[15]  Sylvie Ratté,et al.  Computer-based evaluation of Alzheimer’s disease and mild cognitive impairment patients during a picture description task , 2018, Alzheimer's & dementia.

[16]  Heidi Christensen,et al.  An Avatar-Based System for Identifying Individuals Likely to Develop Dementia , 2017, INTERSPEECH.

[17]  P. S. Mathuranath,et al.  Instrumental activities of daily living scale for dementia screening in elderly people , 2005, International Psychogeriatrics.

[18]  R. Mayeux,et al.  Natural history of Alzheimer’s disease , 2001, Aging.

[19]  Marcos Faúndez-Zanuy,et al.  On Automatic Diagnosis of Alzheimer’s Disease Based on Spontaneous Speech Analysis and Emotional Temperature , 2013, Cognitive Computation.

[20]  B. Croisile,et al.  Comparative Study of Oral and Written Picture Description in Patients with Alzheimer's Disease , 1996, Brain and Language.

[21]  H. Kirshner,et al.  Primary Progressive Aphasia and Alzheimer’s Disease: Brief History, Recent Evidence , 2012, Current Neurology and Neuroscience Reports.

[22]  Heidi Christensen,et al.  Detecting Signs of Dementia Using Word Vector Representations , 2018, INTERSPEECH.

[23]  Jan Alexandersson,et al.  Fully Automatic Speech-Based Analysis of the Semantic Verbal Fluency Task , 2018, Dementia and Geriatric Cognitive Disorders.

[24]  H. Christensen,et al.  Toward the Automation of Diagnostic Conversation Analysis in Patients with Memory Complaints. , 2017, Journal of Alzheimer's disease : JAD.

[25]  Björn W. Schuller,et al.  The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing , 2016, IEEE Transactions on Affective Computing.

[26]  O. Parsons,et al.  Neuropsychology of dementia. , 1986, Neurologic clinics.

[27]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[28]  I. T. Draper THE ASSESSMENT OF APHASIA AND RELATED DISORDERS , 1973 .

[29]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[30]  Björn W. Schuller,et al.  Recent developments in openSMILE, the munich open-source multimedia feature extractor , 2013, ACM Multimedia.

[31]  B Dubois,et al.  [Validation of the Short Cognitive Battery (B2C). Value in screening for Alzheimer's disease and depressive disorders in psychiatric practice]. , 2003, L'Encephale.

[32]  P. Garrard,et al.  Connected speech as a marker of disease progression in autopsy-proven Alzheimer’s disease , 2013, Brain : a journal of neurology.

[33]  H. Braak,et al.  Evolution of the neuropathology of Alzheimer's disease , 1996, Acta neurologica Scandinavica. Supplementum.

[34]  Brian Roark,et al.  Spoken Language Derived Measures for Detecting Mild Cognitive Impairment , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[35]  Saturnino Luz,et al.  A Method for Analysis of Patient Speech in Dialogue for Dementia Detection , 2018, ArXiv.

[36]  Colleen Richey,et al.  Aided diagnosis of dementia type through computer-based analysis of spontaneous speech , 2014, CLPsych@ACL.

[37]  DeLiang Wang,et al.  A Feature Study for Classification-Based Speech Separation at Low Signal-to-Noise Ratios , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[38]  R. Maccioni,et al.  The revitalized tau hypothesis on Alzheimer's disease. , 2010, Archives of medical research.

[39]  D. Royall,et al.  The FAB: A frontal assessment battery at bedside , 2001, Neurology.

[40]  J. Hodges,et al.  Is semantic memory consistently impaired early in the course of Alzheimer's disease? Neuroanatomical and diagnostic implications , 1995, Neuropsychologia.

[41]  Frank Rudzicz,et al.  Vector-space topic models for detecting Alzheimer’s disease , 2016, ACL.

[42]  V. Manera,et al.  Automatic speech analysis for the assessment of patients with predementia and Alzheimer's disease , 2015, Alzheimer's & dementia.

[43]  Saturnino Luz,et al.  Longitudinal Monitoring and Detection of Alzheimer's Type Dementia from Spontaneous Speech Data , 2017, 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS).

[44]  T. Parsons,et al.  NEUROCOGNITIVE DIFFERENTIAL DIAGNOSIS OF DEMENTING DISEASES: ALZHEIMER'S DEMENTIA, VASCULAR DEMENTIA, FRONTOTEMPORAL DEMENTIA, AND MAJOR DEPRESSIVE DISORDER , 2006, The International journal of neuroscience.

[45]  Kristine Yaffe,et al.  Potential for primary prevention of Alzheimer's disease: an analysis of population-based data , 2014, The Lancet Neurology.

[46]  Björn Schuller,et al.  Computational Paralinguistics , 2013 .

[47]  P. Rabins,et al.  Dementia , 2008, Annals of Internal Medicine.

[48]  C. Ritchie,et al.  Protocol for a conversation-based analysis study: PREVENT-ED investigates dialogue features that may help predict dementia onset in later life , 2019, BMJ Open.

[49]  Kathleen C. Fraser,et al.  Linguistic Features Identify Alzheimer's Disease in Narrative Speech. , 2015, Journal of Alzheimer's disease : JAD.

[50]  P. Robert,et al.  Validation de la batterie cognitive courte (B2C). Intérêt pour le dépistage précoce de la maladie d'Alzheimer et des troubles dépressifs en pratique psychiatrique , 2003 .

[51]  Alexandra König,et al.  Speech-based automatic and robust detection of very early dementia , 2014, INTERSPEECH.

[52]  Karmele López-de-Ipiña,et al.  Multi-class Versus One-Class Classifier in Spontaneous Speech Analysis Oriented to Alzheimer Disease Diagnosis , 2022, Recent Advances in Nonlinear Speech Processing.

[53]  Kathleen C. Fraser,et al.  Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment , 2019, Comput. Speech Lang..

[54]  Jamie Reilly,et al.  Cognition, language, and clinical pathological features of non-Alzheimer's dementias: an overview. , 2010, Journal of communication disorders.

[55]  Tanja Schultz,et al.  Manual and Automatic Transcriptions in Dementia Detection from Speech , 2017, INTERSPEECH.

[56]  Sylvester Olubolu Orimaye,et al.  Predicting probable Alzheimer’s disease using linguistic deficits and biomarkers , 2017, BMC Bioinformatics.

[57]  DeLiang Wang,et al.  On Ideal Binary Mask As the Computational Goal of Auditory Scene Analysis , 2005, Speech Separation by Humans and Machines.

[58]  Boyd H. Davis,et al.  Finding a balance: The Carolinas Conversation Collection , 2011 .

[59]  Jeffrey L. Cummings,et al.  Speech and language alterations in dementia syndromes: Characteristics and treatment , 1990 .

[60]  Björn Schuller,et al.  Opensmile: the munich versatile and fast open-source audio feature extractor , 2010, ACM Multimedia.