Investigating Visual Features for Cognitive Impairment Detection Using In-the-wild Data

Early detection of dementia has attracted much research interest due to its crucial role in helping people get suitable treatment or care. Video analysis may provide an effective approach for detection, with low cost and effort compared to current expensive and intensive clinical assessments. This paper investigates the use of a range of visual features - eye blink rate (EBR), head turn rate (HTR) and head movement statistical features (HMSF) - for identifying neurodegenerative disorder (ND), mild cognitive impairment (MCI) and functional memory disorder (FMD). These features are used in a noval multiple thresholds approach, which is applied to an in-the-wild video dataset which includes data recorded in a range of challenging environments. A combination of EBR and HTR gives 78 % accuracy in a three-way classification task (ND/MCI/FMD) and 83%, 83% and 92%, respectively, for the two-way classifications ND/MCI, ND/FMD and MCI/FMD. These results are comparable to related work that uses more features from different modalities. They also provide evidence to support the possibility of an in-the-home detection process for dementia or cognitive impairment.

[1]  H. Christensen,et al.  Eye Blink Rate Based Detection of Cognitive Impairment Using In-the-wild Data , 2021, 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII).

[2]  Guodong Guo,et al.  Visually Interpretable Representation Learning for Depression Recognition from Facial Images , 2020, IEEE Transactions on Affective Computing.

[3]  Satoshi Nakamura,et al.  Detecting Dementia from Face in Human-Agent Interaction , 2019, ICMI.

[4]  Kathleen C. Fraser,et al.  Predicting MCI Status From Multimodal Language Data Using Cascaded Classifiers , 2019, Front. Aging Neurosci..

[5]  Mohammad Soleymani,et al.  AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition , 2019, AVEC@MM.

[6]  Peng Huang,et al.  Learning content‐adaptive feature pooling for facial depression recognition in videos , 2019, Electronics Letters.

[7]  Guodong Guo,et al.  Automated Depression Diagnosis Based on Deep Networks to Encode Facial Appearance and Dynamics , 2018, IEEE Transactions on Affective Computing.

[8]  Fan Zhang,et al.  Artificial Intelligent System for Automatic Depression Level Analysis Through Visual and Vocal Expressions , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[9]  Louis-Philippe Morency,et al.  OpenFace 2.0: Facial Behavior Analysis Toolkit , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[10]  A. Larner,et al.  Attended with and head-turning sign can be clinical markers of cognitive impairment in older adults , 2018, International Psychogeriatrics.

[11]  Satoshi Nakamura,et al.  Detecting Dementia Through Interactive Computer Avatars , 2017, IEEE Journal of Translational Engineering in Health and Medicine.

[12]  Heidi Christensen,et al.  An Avatar-Based System for Identifying Individuals Likely to Develop Dementia , 2017, INTERSPEECH.

[13]  K. Green.  Types of Dementia , 2016 .

[14]  L. Colzato,et al.  Spontaneous eye blink rate as predictor of dopamine-related cognitive function—A review , 2016, Neuroscience & Biobehavioral Reviews.

[15]  Satoshi Nakamura,et al.  Automatic detection of very early stage of dementia through multimodal interaction with computer avatars , 2016, ICMI.

[16]  Michael Wagner,et al.  Cross-cultural detection of depression from nonverbal behaviour , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[17]  Fan Zhang,et al.  Automatic Depression Scale Prediction using Facial Expression Dynamics and Regression , 2014, AVEC '14.

[18]  Hugo Jair Escalante,et al.  Fusing Affective Dimensions and Audio-Visual Features from Segmented Video for Depression Recognition: INAOE-BUAP's Participation at AVEC'14 Challenge , 2014, AVEC '14.

[19]  Björn W. Schuller,et al.  AVEC 2014: 3D Dimensional Affect and Depression Recognition Challenge , 2014, AVEC '14.

[20]  Albert Ali Salah,et al.  Ensemble CCA for Continuous Emotion Prediction , 2014, AVEC '14.

[21]  Panagiotis Bamidis,et al.  Eye Blink Rate as a biological marker of Mild Cognitive Impairment. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[22]  Michael Wagner,et al.  Head Pose and Movement Analysis as an Indicator of Depression , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[23]  Michael Wagner,et al.  Eye movement analysis for depression detection , 2013, 2013 IEEE International Conference on Image Processing.

[24]  Margaret Thorogood,et al.  Prevalences of dementia and cognitive impairment among older people in sub-Saharan Africa: a systematic review. , 2013, Bulletin of the World Health Organization.

[25]  A. Larner.  Head turning sign: pragmatic utility in clinical diagnosis of cognitive impairment , 2012, Journal of Neurology, Neurosurgery & Psychiatry.

[26]  A. Beck,et al.  Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. , 1996, Journal of personality assessment.

[27]  J. Pedersen,et al.  An ethological description of depression , 1988, Acta psychiatrica Scandinavica.

[28]  Cristina Conati,et al.  Non-Invasive Classification of Alzheimer's Disease Using Eye Tracking and Language , 2020, MLHC.

[29]  Tereza Soukupová,et al.  Eye-Blink Detection Using Facial Landmarks , 2016 .

[30]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[31]  R S Baker,et al.  Age-related changes in human blinks. Passive and active changes in eyelid kinematics. , 1997, Investigative ophthalmology & visual science.