Machine Learning for Enhancing Dementia Screening in Ageing Deaf Signers of British Sign Language

Ageing trend in populations is correlated with increased prevalence of acquired cognitive impairments such as dementia. Although there is no cure for dementia, a timely diagnosis helps in obtaining necessary support and appropriate medication. With this in mind, researchers are working urgently to develop effective technological tools that can help doctors undertake early identification of cognitive disorder. In this paper, we introduce an automatic dementia screening system for ageing Deaf signers of British Sign Language (BSL), using Convolutional Neural Networks (CNN), by analysing the sign space envelope and facial expression of BSL signers using normal 2D videos from BSL corpus. Our approach firstly establishes an accurate real-time hand trajectory tracking model together with a real-time landmark facial motion analysis model to identify differences in sign space envelope and facial movement as the keys to identifying language changes associated with dementia. Based on the differences in patterns obtained from facial and trajectory motion data, CNN models (ResNet50/VGG16) are fine-tuned using Keras deep learning models to incrementally identify and improve dementia recognition rates. We report the results for two methods using different modalities (sign trajectory and facial motion), together with the performance comparisons between different deep learning CNN models in ResNet50 and VGG16. The experiments show the effectiveness of our deep learning based approach in terms of sign space tracking, facial motion tracking and early stage dementia performance assessment tasks. The results are validated against cognitive assessment scores as of our ground truth data with a test set performance of 87.88%. The proposed system has potential for economical, simple, flexible, and adaptable assessment of other acquired neurological impairments associated with motor changes, such as stroke and Parkinson’s disease in both hearing and Deaf people.

[1]  Philippe Robert,et al.  PRAXIS: Towards automatic cognitive assessment using gesture recognition , 2018, Expert Syst. Appl..

[2]  Pietro Liò,et al.  A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer's disease , 2018, NeuroImage.

[3]  Bencie Woll,et al.  When sign language breaks down: deaf people's access to language therapy in the UK , 2002 .

[4]  Emanuele Trucco,et al.  Machine learning of neuroimaging to diagnose cognitive impairment and dementia: a systematic review and comparative analysis , 2018, ArXiv.

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Sterling C. Johnson,et al.  Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images , 2017, ArXiv.

[7]  S. R. Bhagyashree,et al.  Diagnosis of Dementia by Machine learning methods in Epidemiological studies: a pilot exploratory study from south India , 2017, Social Psychiatry and Psychiatric Epidemiology.

[8]  Masashi Kameyama,et al.  Deep learning-based imaging classification identified cingulate island sign in dementia with Lewy bodies , 2019 .

[9]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Daniel Austin,et al.  In-home walking speeds and variability trajectories associated with mild cognitive impairment , 2012, Neurology.

[13]  Bencie Woll,et al.  Real Time Hand Movement Trajectory Tracking for Enhancing Dementia Screening in Ageing Deaf Signers of British Sign Language , 2019, CD-MAKE.