Cross-lingual detection of mild cognitive impairment based on temporal parameters of spontaneous speech

Abstract Mild Cognitive Impairment (MCI) is a heterogeneous clinical syndrome, often considered as the prodromal stage of dementia. It is characterized by the subtle deterioration of cognitive functions, including memory, executive functions and language. Mainly due to the tenuous nature of these impairments, a high percentage of MCI cases remain undetected. There is evidence that language changes in MCI are present even before the manifestation of other distinctive cognitive symptoms, which offers a chance for early recognition. A cheap non-invasive way of early screening could be the use of automatic speech analysis. Earlier, our research team developed a set of speech temporal parameters, and demonstrated its applicability for MCI detection. For the automatic extraction of these attributes, a Hungarian-language ASR system was employed to match the native language of the MCI and healthy control (HC) subjects. In practical applications, however, it would be convenient to use exactly the same tool, regardless of the language spoken by the subjects. In this study we show that our temporal parameter set, consisting of articulation rate, speech tempo and various other attributes describing the hesitation of the subject, can indeed be reliably extracted regardless of the language of the ASR system used. For this purpose, we performed experiments both on English-speaking and on Hungarian-speaking MCI patients and healthy control subjects, using English and Hungarian ASR systems in both cases. Our experimental results indicate that the language on which the ASR system was trained only slightly affects the MCI classification performance, because we got quite similar scores (67-92%) as we did in the monolingual cases (67-92% as well). As our last investigation, we compared the proposed attribute values for the same utterances, utilizing both the English and the Hungarian ASR models. We found that the articulation rate and speech tempo values calculated based on the two ASR models were highly correlated, and so were the attributes corresponding to silent pauses; however, noticeable differences were found regarding the filled pauses (still, these attributes remained indicative for both languages). Our further analysis revealed that this is probably due to a difference regarding the annotation of the English and the Hungarian ASR training utterances.

[1]  Sterling C. Johnson,et al.  Declines in Connected Language Are Associated with Very Early Mild Cognitive Impairment: Results from the Wisconsin Registry for Alzheimer’s Prevention , 2018, Front. Aging Neurosci..

[2]  A. Al-Ghazali,et al.  Silent Pauses in the Speech of Yemeni EFL Learners , 2019, ELS Journal on Interdisciplinary Studies in Humanities.

[3]  H. Brodaty,et al.  ALZHEIMER'S DISEASE INTERNATIONAL , 1997, International journal of geriatric psychiatry.

[4]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[5]  Niko Brümmer,et al.  The BOSARIS Toolkit: Theory, Algorithms and Code for Surviving the New DCF , 2013, ArXiv.

[6]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[7]  Alexandra König,et al.  Speech-based automatic and robust detection of very early dementia , 2014, INTERSPEECH.

[8]  L. Fratiglioni,et al.  Mild cognitive impairment: a concept in evolution , 2014, Journal of internal medicine.

[9]  Kathleen C. Fraser,et al.  Predicting MCI Status From Multimodal Language Data Using Cascaded Classifiers , 2019, Front. Aging Neurosci..

[10]  Yiannis Kompatsiaris,et al.  Evaluation of speech-based protocol for detection of early-stage dementia , 2013, INTERSPEECH.

[11]  J. Weuve,et al.  2016 Alzheimer's disease facts and figures , 2016 .

[12]  M. Prince,et al.  World Alzheimer Report 2015 - The Global Impact of Dementia: An analysis of prevalence, incidence, cost and trends , 2015 .

[13]  Jing Huang,et al.  Towards automatic transcription of large spoken archives - English ASR for the MALACH project , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[14]  Fabio Tamburini,et al.  Speech Analysis by Natural Language Processing Techniques: A Possible Tool for Very Early Detection of Cognitive Decline? , 2018, Front. Aging Neurosci..

[15]  Dimitrios Kokkinakis,et al.  Identification of Mild Cognitive Impairment From Speech in Swedish Using Deep Sequential Neural Networks , 2018, Front. Neurol..

[16]  Shunichi Ishihara,et al.  Likelihood Ratio-based Forensic Voice Comparison on L2 speakers: A Case of Hong Kong native male production of English vowels , 2015, ALTA.

[17]  Gábor Gosztolya,et al.  A Speech Recognition-based Solution for the Automatic Detection of Mild Cognitive Impairment from Spontaneous Speech , 2018, Current Alzheimer research.

[18]  Patrick Kenny,et al.  Front-End Factor Analysis for Speaker Verification , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  László Tóth Phone recognition with hierarchical convolutional deep maxout networks , 2015, EURASIP J. Audio Speech Music. Process..

[20]  Anup D. Patel,et al.  Quality improvement in neurology , 2015, Neurology.

[21]  Elmar Nöth,et al.  Multimodal I-vectors to Detect and Evaluate Parkinson's Disease , 2018, INTERSPEECH.

[22]  V. Manera,et al.  Automatic speech analysis for the assessment of patients with predementia and Alzheimer's disease , 2015, Alzheimer's & dementia.

[23]  P J Manos,et al.  The Ten Point Clock Test: A Quick Screen and Grading Method for Cognitive Impairment in Medical and Surgical Patients , 1994, International journal of psychiatry in medicine.

[24]  C. Arsene,et al.  Quality improvement. , 2015, Nursing management.

[25]  Steve Young,et al.  The HTK book , 1995 .

[26]  Dimitrios Kokkinakis,et al.  Voice quality and speech fluency distinguish individuals with Mild Cognitive Impairment from Healthy Controls , 2020, PloS one.

[27]  J. Yesavage,et al.  Geriatric Depression Scale (GDS): Recent evidence and development of a shorter version. , 1986 .

[28]  Björn W. Schuller,et al.  Spotting Social Signals in Conversational Speech over IP: A Deep Learning Perspective , 2017, INTERSPEECH.

[29]  Frank Rudzicz,et al.  Automatic speech recognition in the diagnosis of primary progressive aphasia , 2013, SLPAT.

[30]  Paul Deléglise,et al.  TED-LIUM: an Automatic Speech Recognition dedicated corpus , 2012, LREC.

[31]  András Beke,et al.  Development of a Large Spontaneous Speech Database of Agglutinative Hungarian Language , 2014, TSD.

[32]  Meysam Asgari,et al.  Predicting mild cognitive impairment from spontaneous spoken utterances , 2017, Alzheimer's & dementia.

[33]  A. Back,et al.  An Automated Approach to Examining Pausing in the Speech of People With Dementia , 2020, American journal of Alzheimer's disease and other dementias.

[34]  Alexandra Konig,et al.  Use of Speech Analyses within a Mobile Application for the Assessment of Cognitive Impairment in Elderly People. , 2018, Current Alzheimer research.

[35]  Simon Lovestone,et al.  Is MCI really just early dementia? A systematic review of conversion studies , 2004, International Psychogeriatrics.

[36]  Antanas Verikas,et al.  Detecting Parkinson’s disease from sustained phonation and speech signals , 2017, PloS one.

[37]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[38]  Marcin Witkowski,et al.  Structure of pauses in speech in the context of speaker verification and classification of speech type , 2016, EURASIP J. Audio Speech Music. Process..

[39]  Elmar Nöth,et al.  Automatic Diagnosis of Alzheimer’s Disease Using Neural Network Language Models , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  J. Shega,et al.  Quality improvement in neurology , 2019, Neurology.

[41]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[42]  Yogesan Kanagasingam,et al.  Innovative diagnostic tools for early detection of Alzheimer's disease , 2015, Alzheimer's & Dementia.

[43]  V. Leirer,et al.  Development and validation of a geriatric depression screening scale: a preliminary report. , 1982, Journal of psychiatric research.

[44]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[45]  Satoshi Nakamura,et al.  Cross-lingual Speech-based Tobi Label Generation Using Bidirectional Lstm , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[46]  Pilar Sanz,et al.  On the analysis of speech and disfluencies for automatic detection of Mild Cognitive Impairment , 2020, Neural Comput. Appl..

[47]  Juan Ignacio Godino-Llorente,et al.  Analysis of speaker recognition methodologies and the influence of kinetic changes to automatically detect Parkinson's Disease , 2018, Appl. Soft Comput..

[48]  Nicholas W. D. Evans,et al.  Preserving privacy in speaker and speech characterisation , 2019, Comput. Speech Lang..

[49]  Kathleen C. Fraser,et al.  Automated classification of primary progressive aphasia subtypes from narrative speech transcripts , 2014, Cortex.

[50]  Gábor Gosztolya,et al.  Automatic detection of mild cognitive impairment from spontaneous speech using ASR , 2015, INTERSPEECH.

[51]  E. Tangalos,et al.  Mild Cognitive Impairment Clinical Characterization and Outcome , 1999 .

[52]  Amy E. Sanders,et al.  Quality improvement in neurology , 2018, Neurology.

[53]  Julien Epps,et al.  Estimating the Precision of the Likelihood-Ratio Output of a Forensic-Voice-Comparison System , 2010, Odyssey.

[54]  Gábor Gosztolya,et al.  Identifying Mild Cognitive Impairment and mild Alzheimer's disease based on spontaneous speech using ASR and linguistic features , 2019, Comput. Speech Lang..

[55]  Gábor Gosztolya,et al.  Detecting Mild Cognitive Impairment from Spontaneous Speech by Correlation-Based Phonetic Feature Selection , 2016, INTERSPEECH.

[56]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[57]  Esther de Leeuw,et al.  Hesitation Markers in English, German, and Dutch , 2007 .

[58]  Veronika Vincze,et al.  Speaking in Alzheimer’s Disease, is That an Early Sign? Importance of Changes in Language Abilities in Alzheimer’s Disease , 2015, Front. Aging Neurosci..