Assessing visual attributes of handwriting for prediction of neurological disorders - A case study on Parkinson's disease

Abstract Parkinson’s disease (PD) is a degenerative disorder that progressively affects the central nervous system causing muscle rigidity, tremors, slowed movements and impaired balance. Sophisticated diagnostic procedures like SPECT scans can detect changes in the brain caused by PD but are only effective once the disease has advanced considerably. Analysis of subtle variations in handwriting and speech can serve as potential tools for early prediction of the disease. While traditional techniques mostly rely on dynamic (kinematic and spatio-temporal) features of handwriting, in this study, we quantitatively evaluate the visual attributes in characterization of graphomotor samples of PD patients. For this purpose, Convolutional Neural Networks are employed to extract discriminating visual features from multiple representations of various graphomotor samples produced by both control and PD subjects. The extracted features are then fed to a Support Vector Machine (SVM) classifier. Evaluations are carried out on a dataset of 72 subjects using early and late fusion techniques and an overall accuracy of 83% is realized with solely visual information.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  H. Teulings,et al.  Advances in graphonomics: studies on fine motor control, its development and disorders. , 2006, Human movement science.

[3]  Ana Luisa Trejos,et al.  The measurement and analysis of Parkinsonian hand tremor , 2016, 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI).

[4]  Gesine L. Alders,et al.  Kinematic analysis of dopaminergic effects on skilled handwriting movements in Parkinson’s disease , 2006, Journal of Neural Transmission.

[5]  G. Stelmach,et al.  Parkinsonism Reduces Coordination of Fingers, Wrist, and Arm in Fine Motor Control , 1997, Experimental Neurology.

[6]  Zdenek Smekal,et al.  Decision Support Framework for Parkinson’s Disease Based on Novel Handwriting Markers , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  Ching Y. Suen,et al.  A trainable feature extractor for handwritten digit recognition , 2007, Pattern Recognit..

[8]  Clayton R. Pereira,et al.  Deep Learning-Aided Parkinson's Disease Diagnosis from Handwritten Dynamics , 2016, 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[9]  Tim Lüth,et al.  Quantitative evaluation of Parkinson's disease using sensor based smart glove , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[10]  K. Feder,et al.  Handwriting development, competency, and intervention , 2007, Developmental medicine and child neurology.

[11]  U Hegerl,et al.  Kinematic analysis of handwriting movements in patients with obsessive-compulsive disorder , 2001, Journal of neurology, neurosurgery, and psychiatry.

[12]  Tom Chau,et al.  Handwriting Difficulties in Children with Autism Spectrum Disorders: A Scoping Review , 2011, Journal of autism and developmental disorders.

[13]  Luca Palmerini,et al.  Feature Selection for Accelerometer-Based Posture Analysis in Parkinson's Disease , 2011, IEEE Transactions on Information Technology in Biomedicine.

[14]  D. Ruta,et al.  An Overview of Classifier Fusion Methods , 2000 .

[15]  R. Iman,et al.  Approximations of the critical region of the fbietkan statistic , 1980 .

[16]  Alexander C. Berg,et al.  Combining multiple sources of knowledge in deep CNNs for action recognition , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[17]  Sara Rosenblum,et al.  Handwriting process and product characteristics of children diagnosed with developmental coordination disorder. , 2008, Human movement science.

[18]  Mohammad H. Mahoor,et al.  Going deeper in facial expression recognition using deep neural networks , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[20]  S. Chung,et al.  Diagnosis and treatment of hand tremor , 2012 .

[21]  M. Friedman A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[22]  S. Rosenblum,et al.  Age-related changes in executive control and their relationships with activity performance in handwriting. , 2013, Human movement science.

[23]  Marcos Faúndez-Zanuy,et al.  Evaluation of handwriting kinematics and pressure for differential diagnosis of Parkinson's disease , 2016, Artif. Intell. Medicine.

[24]  Zdenek Smekal,et al.  Prediction potential of different handwriting tasks for diagnosis of Parkinson's , 2013, 2013 E-Health and Bioengineering Conference (EHB).

[25]  Ching Y. Suen,et al.  A novel hybrid CNN-SVM classifier for recognizing handwritten digits , 2012, Pattern Recognit..

[26]  H. Möller,et al.  Kinematic Analysis of Handwriting Movements in Patients with Alzheimer’s Disease, Mild Cognitive Impairment, Depression and Healthy Subjects , 2003, Dementia and Geriatric Cognitive Disorders.

[27]  J. Bradshaw,et al.  Consistency of handwriting movements in dementia of the Alzheimer's type: A comparison with Huntington's and Parkinson's diseases , 1999, Journal of the International Neuropsychological Society.

[28]  João Cevada,et al.  ParkDetect: Early diagnosing Parkinson's Disease , 2014, 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[29]  N Mai,et al.  Computational analysis of open loop handwriting movements in Parkinson's disease: A rapid method to detect dopamimetic effects , 1996, Movement disorders : official journal of the Movement Disorder Society.

[30]  Michael I. Miller,et al.  Novel automated morphometric and kinematic handwriting assessment: A validity study in children with ASD and ADHD , 2017 .

[31]  Christine Klein,et al.  Digitized spiral analysis is a promising early motor marker for Parkinson Disease. , 2010, Parkinsonism & Related Disorders.

[32]  Hongwei Liu,et al.  Convolutional Neural Network With Data Augmentation for SAR Target Recognition , 2016, IEEE Geoscience and Remote Sensing Letters.

[33]  Pierre Courtellemont,et al.  Automatic analysis of the structuring of children's drawings and writing , 2002, Pattern Recognit..

[34]  S. L. Smith,et al.  A novel computer-based technique for the assessment of tremor in Parkinson's disease. , 2007, Age and ageing.

[35]  Clayton R. Pereira,et al.  A Step Towards the Automated Diagnosis of Parkinson's Disease: Analyzing Handwriting Movements , 2015, 2015 IEEE 28th International Symposium on Computer-Based Medical Systems.

[36]  Mehrtash Tafazzoli Harandi,et al.  Going deeper into action recognition: A survey , 2016, Image Vis. Comput..

[37]  J Walton Handwriting changes due to aging and Parkinson's syndrome. , 1997, Forensic science international.

[38]  Marie Vidailhet,et al.  Micrographia secondary to lenticular lesions , 2002, Movement disorders : official journal of the Movement Disorder Society.

[39]  S. Fahn Unified Parkinson's Disease Rating Scale , 1987 .

[40]  Ersin Yumer,et al.  Shape Synthesis from Sketches via Procedural Models and Convolutional Networks , 2017, IEEE Transactions on Visualization and Computer Graphics.

[41]  A. Bastian,et al.  Children with autism show specific handwriting impairments , 2009, Neurology.

[42]  G. Stelmach,et al.  Adaptation of handwriting size under distorted visual feedback in patients with Parkinson's disease and elderly and young controls , 2002, Journal of neurology, neurosurgery, and psychiatry.

[43]  Natasha M. Maurits,et al.  Standardized Handwriting to Assess Bradykinesia, Micrographia and Tremor in Parkinson's Disease , 2014, PloS one.

[44]  M. Hallett,et al.  Pathophysiology of bradykinesia in Parkinson's disease. , 2001, Brain : a journal of neurology.

[45]  M. Patterson,et al.  Spiral analysis in Niemann‐Pick disease type C , 2009, Movement disorders : official journal of the Movement Disorder Society.

[46]  Rifat Sipahi,et al.  Objective Quantitative Assessment of Movement Disorders Through Analysis of Static Handwritten Characters , 2015 .

[47]  E. Louis,et al.  The spiral axis as a clinical tool to distinguish essential tremor from dystonia cases. , 2014, Parkinsonism & Related Disorders.

[48]  M. Breteler,et al.  Epidemiology of Parkinson's disease , 2006, The Lancet Neurology.

[49]  M. Samuel,et al.  Handwriting as an objective tool for Parkinson’s disease diagnosis , 2013, Journal of Neurology.

[50]  Jirí Mekyska,et al.  A new modality for quantitative evaluation of Parkinson's disease: In-air movement , 2013, 13th IEEE International Conference on BioInformatics and BioEngineering.

[51]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[52]  S L Pullman,et al.  Spiral Analysis: A New Technique for Measuring Tremor With a Digitizing Tablet , 2008, Movement disorders : official journal of the Movement Disorder Society.

[53]  K. Bötzel,et al.  Prevalence and incidence of Parkinson's disease in Europe , 2005, European Neuropsychopharmacology.

[54]  R. Lipton,et al.  Validity of spiral analysis in early Parkinson's disease , 2008, Movement disorders : official journal of the Movement Disorder Society.

[55]  M. Szarvas,et al.  Pedestrian detection with convolutional neural networks , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[56]  Imran Siddiqi,et al.  Automated scoring of Bender Gestalt Test using image analysis techniques , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[57]  Marcos Faúndez-Zanuy,et al.  Analysis of in-air movement in handwriting: A novel marker for Parkinson's disease , 2014, Comput. Methods Programs Biomed..

[58]  João Paulo Papa,et al.  Classification of handwriting patterns in patients with Parkinson´s disease, using a biometric sensor , 2014 .

[59]  Max A. Little,et al.  Accurate Telemonitoring of Parkinson's Disease Progression by Noninvasive Speech Tests , 2009, IEEE Transactions on Biomedical Engineering.

[60]  Jin H. Yan,et al.  Alzheimer's disease and mild cognitive impairment deteriorate fine movement control. , 2008, Journal of psychiatric research.

[61]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[62]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[63]  Beth Patricia Johnson,et al.  Do children with autism and Asperger's disorder have difficulty controlling handwriting size? A kinematic evaluation , 2015 .

[64]  Stephan P. Swinnen,et al.  Relearning of writing skills in Parkinson's disease: A literature review on influential factors and optimal strategies , 2013, Neuroscience & Biobehavioral Reviews.

[65]  Sara Rosenblum,et al.  Do motor ability and handwriting kinematic measures predict organizational ability among children with Developmental Coordination Disorders? , 2015, Human movement science.

[66]  Michael P. Caligiuri,et al.  The Neuroscience of Handwriting: Applications for Forensic Document Examination , 2012 .

[67]  A. Korczyn,et al.  Handwriting process variables discriminating mild Alzheimer's disease and mild cognitive impairment. , 2006, The journals of gerontology. Series B, Psychological sciences and social sciences.