Towards Automated and Marker-less Parkinson Disease Assessment: Predicting UPDRS Scores using Sit-stand videos

This paper presents a novel deep learning enabled, video based analysis framework for assessing the Unified Parkinson’s Disease Rating Scale (UPDRS) that can be used in the clinic or at home. We report results from comparing the performance of the framework to that of trained clinicians on a population of 32 Parkinson’s disease (PD) patients. In-person clinical assessments by trained neurologists are used as the ground truth for training our framework and for comparing the performance. We find that the standard sit-to-stand activity can be used to evaluate the UPDRS sub-scores of bradykinesia (BRADY) and posture instability and gait disorders (PIGD). For BRADY we find Fl-scores of 0.75 using our framework compared to 0.50 for the video based rater clinicians, while for PIGD we find 0.78 for the framework and 0.45 for the video based rater clinicians. We believe our proposed framework has potential to provide clinically acceptable end points of PD in greater granularity without imposing burdens on patients and clinicians, which empowers a variety of use cases such as passive tracking of PD progression in spaces such as nursing homes, in-home self-assessment, and enhanced tele-medicine.

[1]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[2]  R. Hauser,et al.  Parkinson's disease home diary: Further validation and implications for clinical trials , 2004, Movement disorders : official journal of the Movement Disorder Society.

[3]  Tieniu Tan,et al.  An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  P. Olivier,et al.  Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease. , 2014, Gait & posture.

[5]  Jefersson Alex dos Santos,et al.  SkeleMotion: A New Representation of Skeleton Joint Sequences based on Motion Information for 3D Action Recognition , 2019, 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[6]  Vittorio Caggiano,et al.  Using an unbiased symbolic movement representation to characterize Parkinson’s disease states , 2020, Scientific Reports.

[7]  Austin Reiter,et al.  Interpretable 3D Human Action Analysis with Temporal Convolutional Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Gang Wang,et al.  Skeleton-Based Human Action Recognition With Global Context-Aware Attention LSTM Networks , 2017, IEEE Transactions on Image Processing.

[9]  Jianbin Tang,et al.  SSHFD: Single Shot Human Fall Detection with Occluded Joints Resilience , 2020, ECAI.

[10]  P. Bonato,et al.  mHealth and wearable technology should replace motor diaries to track motor fluctuations in Parkinson’s disease , 2020, npj Digital Medicine.

[11]  Academisch Proefschrift,et al.  UvA-DARE ( Digital Academic Repository ) Clinimetrics , clinical profile and prognosis in early Parkinson ’ s disease , 2009 .

[12]  C. Demanuele,et al.  Development of digital biomarkers for resting tremor and bradykinesia using a wrist-worn wearable device , 2020, npj Digital Medicine.

[13]  Geoffrey E. Hinton,et al.  Regularizing Neural Networks by Penalizing Confident Output Distributions , 2017, ICLR.

[14]  Geoffrey E. Hinton,et al.  When Does Label Smoothing Help? , 2019, NeurIPS.

[15]  John Cosmas,et al.  Recognition of postures and Freezing of Gait in Parkinson's disease patients using Microsoft Kinect sensor , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[16]  Moataz Eltoukhy,et al.  Microsoft Kinect can distinguish differences in over-ground gait between older persons with and without Parkinson's disease. , 2017, Medical engineering & physics.

[17]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction , 2018, ArXiv.

[18]  Jorge Cancela,et al.  Wearable sensor network for health monitoring: the case of Parkinson disease , 2013 .

[19]  Daniil Osokin,et al.  Real-time 2D Multi-Person Pose Estimation on CPU: Lightweight OpenPose , 2018, ICPRAM.

[20]  A. Ascherio,et al.  The epidemiology of Parkinson's disease: risk factors and prevention , 2016, The Lancet Neurology.

[21]  艾而帝,et al.  Microsoft Kinect 虛擬復健系統設計 , 2013 .

[22]  Chao Li,et al.  Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation , 2018, IJCAI.