To BERT or Not To BERT: Comparing Speech and Language-based Approaches for Alzheimer's Disease Detection

Research related to automatically detecting Alzheimer's disease (AD) is important, given the high prevalence of AD and the high cost of traditional methods. Since AD significantly affects the content and acoustics of spontaneous speech, natural language processing and machine learning provide promising techniques for reliably detecting AD. We compare and contrast the performance of two such approaches for AD detection on the recent ADReSS challenge dataset: 1) using domain knowledge-based hand-crafted features that capture linguistic and acoustic phenomena, and 2) fine-tuning Bidirectional Encoder Representations from Transformer (BERT)-based sequence classification models. We also compare multiple feature-based regression models for a neuropsychological score task in the challenge. We observe that fine-tuned BERT models, given the relative importance of linguistics in cognitive impairment detection, outperform feature-based approaches on the AD detection task.

[1]  O. Kinouchi,et al.  Speech Graphs Provide a Quantitative Measure of Thought Disorder in Psychosis , 2012, PloS one.

[2]  Kathleen C. Fraser,et al.  Linguistic Features Identify Alzheimer's Disease in Narrative Speech. , 2015, Journal of Alzheimer's disease : JAD.

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  Boyd H. Davis,et al.  Examining Pauses in Alzheimer's Discourse , 2009, American journal of Alzheimer's disease and other dementias.

[5]  Amy Beth Warriner,et al.  Norms of valence, arousal, and dominance for 13,915 English lemmas , 2013, Behavior Research Methods.

[6]  M. Prince,et al.  World Alzheimer report 2016: improving healthcare for people living with dementia: coverage, quality and costs now and in the future , 2016 .

[7]  Mirella Lapata,et al.  Text Summarization with Pretrained Encoders , 2019, EMNLP.

[8]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[9]  J. Becker,et al.  The natural history of Alzheimer's disease. Description of study cohort and accuracy of diagnosis. , 1994, Archives of neurology.

[10]  Frank Rudzicz,et al.  On the importance of normative data in speech-based assessment , 2017, ArXiv.

[11]  Erik Cambria,et al.  Recent Trends in Deep Learning Based Natural Language Processing , 2017, IEEE Comput. Intell. Mag..

[12]  Benoît Sagot,et al.  What Does BERT Learn about the Structure of Language? , 2019, ACL.

[13]  Jekaterina Novikova,et al.  Detecting cognitive impairments by agreeing on interpretations of linguistic features , 2018, NAACL.

[14]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[15]  Xiaofei Lu,et al.  Automatic analysis of syntactic complexity in second language writing , 2010 .

[16]  Mohit Bansal,et al.  Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models , 2018, NAACL.

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[19]  L. Gise,et al.  Principles and Practice of Geriatric Psychiatry , 2007, Annals of Internal Medicine.

[20]  Ani Nenkova,et al.  Predicting the Fluency of Text with Shallow Structural Features: Case Studies of Machine Translation and Human-Written Text , 2009, EACL.

[21]  B. MacWhinney The Childes Project: Tools for Analyzing Talk, Volume I: Transcription format and Programs , 2000 .

[22]  Mark Dredze,et al.  Combining Word Embeddings and Feature Embeddings for Fine-grained Relation Extraction , 2015, HLT-NAACL.

[23]  Sylvester Olubolu Orimaye,et al.  Learning Linguistic Biomarkers for Predicting Mild Cognitive Impairment using Compound Skip-grams , 2015, ArXiv.

[24]  Jekaterina Novikova,et al.  Semi-supervised classification by reaching consensus among modalities , 2018, ArXiv.

[25]  J. Escudero,et al.  Machine-learning based identification of undiagnosed dementia in primary care: a feasibility study , 2018, BJGP open.

[26]  G. Prabhakaran,et al.  Analysis of Structure and Cost in a Longitudinal Study of Alzheimer’s Disease , 2018 .

[27]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[28]  C. Derouesné [Mini-mental state examination]. , 2001, Revue neurologique.

[29]  Fasih Haider,et al.  Alzheimer's Dementia Recognition through Spontaneous Speech: The ADReSS Challenge , 2020, INTERSPEECH.

[30]  Frank Rudzicz,et al.  Learning multiview embeddings for assessing dementia , 2018, EMNLP.

[31]  Ann Bies,et al.  The Penn Treebank: Annotating Predicate Argument Structure , 1994, HLT.

[32]  Jekaterina Novikova,et al.  The Effect of Heterogeneous Data for Alzheimer's Disease Detection from Speech , 2018, ArXiv.

[33]  B. Croisile,et al.  Comparative Study of Oral and Written Picture Description in Patients with Alzheimer's Disease , 1996, Brain and Language.

[34]  Frank Rudzicz,et al.  Using linguistic features longitudinally to predict clinical scores for Alzheimer’s disease and related dementias , 2015, SLPAT@Interspeech.

[35]  R'emi Louf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.