Classification of Alzheimer's using Deep-learning Methods on Webcam-based Gaze Data

There has been increasing interest in non-invasive predictors of Alzheimer's disease (AD) as an initial screen for this condition. Previously, successful attempts leveraged eye-tracking and language data generated during picture narration and reading tasks. These results were obtained with high-end, expensive eye-trackers. Instead, we explore classification using eye-tracking data collected with a webcam, where our classifiers are built using a deep-learning approach. Our results show that the webcam gaze classifier is not as good as the classifier based on high-end eye-tracking data. However, the webcam-based classifier still beats the majority-class baseline classifier in terms of AU-ROC, indicating that predictive signals can be extracted from webcam gaze tracking. Hence, although our results indicate that there is still a long way to go before webcam gaze tracking can reach practical relevance, they still provide an encouraging proof of concept that this technology should be further explored as an affordable alternative to high-end eye-trackers for the detection of AD.

[1]  Yi Tay,et al.  Efficient Transformers: A Survey , 2020, ACM Comput. Surv..

[2]  S. D’Mello,et al.  Evaluating Calibration-free Webcam-based Eye Tracking for Gaze-based User Modeling , 2022, ICMI.

[3]  Taylan K. Sen,et al.  Are You Really Looking at Me? A Feature-Extraction Framework for Estimating Interpersonal Eye Gaze From Conventional Video , 2019, IEEE Transactions on Affective Computing.

[4]  C. Conati,et al.  Classification of Alzheimer’s Disease Leveraging Multi-task Machine Learning Analysis of Speech and Eye-Movement Data , 2021, Frontiers in Human Neuroscience.

[5]  Jack Mostow,et al.  Early Prediction of Children's Disengagement in a Tablet Tutor Using Visual Features , 2021, AIED.

[6]  Cristina Conati,et al.  A Neural Architecture for Detecting User Confusion in Eye-tracking Data , 2020, ICMI.

[7]  Junfeng He,et al.  Accelerating eye movement research via accurate and affordable smartphone eye tracking , 2020, Nature Communications.

[8]  Cristina Conati,et al.  Non-Invasive Classification of Alzheimer's Disease Using Eye Tracking and Language , 2020, MLHC.

[9]  Louise Cummings,et al.  Describing the Cookie Theft picture , 2019, Pragmatics and Society.

[10]  Giuseppe Carenini,et al.  A Neural Model for Predicting Dementia from Language , 2019, MLHC.

[11]  Xucong Zhang,et al.  Robust eye contact detection in natural multi-person interactions using gaze and speaking behaviour , 2018, ETRA.

[12]  Louis-Philippe Morency,et al.  OpenFace 2.0: Facial Behavior Analysis Toolkit , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[13]  Mohit Bansal,et al.  Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models , 2018, NAACL.

[14]  Shuai Li,et al.  Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Yusuke Sugano,et al.  Everyday Eye Contact Detection Using Unsupervised Gaze Target Discovery , 2017, UIST.

[16]  Katherine A. Heller,et al.  An Improved Multi-Output Gaussian Process RNN with Real-Time Validation for Early Sepsis Detection , 2017, MLHC.

[17]  Nick C. Fox,et al.  Eyetracking Metrics in Young Onset Alzheimer’s Disease: A Window into Cognitive Visual Functions , 2017, Front. Neurol..

[18]  Peter Corcoran,et al.  A Review and Analysis of Eye-Gaze Estimation Systems, Algorithms and Performance Evaluation Methods in Consumer Platforms , 2017, IEEE Access.

[19]  Silvia Castro,et al.  Eye-Movement behavior identification for AD diagnosis , 2017, ArXiv.

[20]  Sidney K. D'Mello,et al.  Zone out no more: Mitigating mind wandering during computerized reading , 2017, EDM.

[21]  Florian Schmidt,et al.  Neural Document Embeddings for Intensive Care Patient Mortality Prediction , 2016, NIPS 2016.

[22]  Michael C. Frank,et al.  Vision-Based Classification of Developmental Disorders Using Eye-Movements , 2016, MICCAI.

[23]  Cristina Conati,et al.  Predicting Confusion in Information Visualization from Eye Tracking and Interaction Data , 2016, IJCAI.

[24]  Truong Q. Nguyen,et al.  Context Matters: Refining Object Detection in Video with Recurrent Neural Networks , 2016, BMVC.

[25]  Wojciech Matusik,et al.  Eye Tracking for Everyone , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Tim J Anderson,et al.  Eye movements in neurodegenerative diseases. , 2016, Current opinion in neurology.

[27]  Charles Elkan,et al.  Learning to Diagnose with LSTM Recurrent Neural Networks , 2015, ICLR.

[28]  Kathleen C. Fraser,et al.  Linguistic Features Identify Alzheimer's Disease in Narrative Speech. , 2015, Journal of Alzheimer's disease : JAD.

[29]  S. D’Mello,et al.  Automatic gaze-based user-independent detection of mind wandering during computerized reading , 2016, User Modeling and User-Adapted Interaction.

[30]  Matthew J. Hausknecht,et al.  Beyond short snippets: Deep networks for video classification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Philip C. Ko,et al.  Eye movements in Alzheimer's disease. , 2015, Journal of Alzheimer's disease : JAD.

[32]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[33]  Jochen Laubrock,et al.  Registering eye movements during reading in Alzheimer’s disease: Difficulties in predicting upcoming words , 2014, Journal of clinical and experimental neuropsychology.

[34]  Malaz Boustani,et al.  Alzheimer's Association recommendations for operationalizing the detection of cognitive impairment during the Medicare Annual Wellness Visit in a primary care setting , 2013, Alzheimer's & Dementia.

[35]  S. Trauzettel-Klosinski,et al.  Standardized assessment of reading performance: the New International Reading Speed Texts IReST. , 2012, Investigative ophthalmology & visual science.

[36]  K. Yaffe,et al.  The projected effect of risk factor reduction on Alzheimer's disease prevalence , 2011, The Lancet Neurology.

[37]  B. Miller,et al.  Oculomotor function in frontotemporal lobar degeneration, related disorders and Alzheimer's disease , 2008, Brain : a journal of neurology.

[38]  J. Cummings,et al.  The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool For Mild Cognitive Impairment , 2005, Journal of the American Geriatrics Society.

[39]  I. T. Draper THE ASSESSMENT OF APHASIA AND RELATED DISORDERS , 1973 .