Crowdsourcing digital health measures to predict Parkinson’s disease severity: the Parkinson’s Disease Digital Biomarker DREAM Challenge

Mobile health, the collection of data using wearables and sensors, is a rapidly growing field in health research with many applications. Deriving validated measures of disease and severity that can be used clinically or as outcome measures in clinical trials, referred to as digital biomarkers, has proven difficult. In part due to the complicated analytical approaches necessary to develop these metrics. Here we describe the use of crowdsourcing to specifically evaluate and benchmark features derived from accelerometer and gyroscope data in two different datasets to predict the presence of Parkinson’s Disease (PD) and severity of three PD symptoms: tremor, dyskinesia and bradykinesia. 40 teams from around the world submitted features, and achieved drastically improved predictive performance for PD (best AUROC=0.87), as well as severity of tremor (best AUPR=0.75), dyskinesia (best AUPR=0.48) and bradykinesia (best AUPR=0.95).

[1]  Arno Klein,et al.  Personalized Hypothesis Tests for Detecting Medication Response in Parkinson Disease Patients Using iPhone Sensor Data , 2016, PSB.

[2]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[3]  Ahmad Ihsan Mohd Yassin,et al.  Statistical analysis of parkinson disease gait classification using Artificial Neural Network , 2011, 2011 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[4]  Tjerk P. Straatsma,et al.  NWChem: A comprehensive and scalable open-source solution for large scale molecular simulations , 2010, Comput. Phys. Commun..

[5]  Nir Giladi,et al.  Characterization of freezing of gait subtypes and the response of each to levodopa in Parkinson's disease , 2003, European journal of neurology.

[6]  M. Muenter,et al.  Frequency of levodopa‐related dyskinesias and motor fluctuations as estimated from the cumulative literature , 2001, Movement disorders : official journal of the Movement Disorder Society.

[7]  Suchi Saria,et al.  Using Smartphones and Machine Learning to Quantify Parkinson Disease Severity: The Mobile Parkinson Disease Score , 2018, JAMA neurology.

[8]  D. Jennings,et al.  The natural history of the syndrome of primary progressive freezing gait. , 2002, Archives of neurology.

[9]  Juan Carlos Fernández,et al.  Multiobjective evolutionary algorithms to identify highly autocorrelated areas: the case of spatial distribution in financially compromised farms , 2014, Ann. Oper. Res..

[10]  Max Kuhn,et al.  Building Predictive Models in R Using the caret Package , 2008 .

[11]  R. Fitzpatrick,et al.  The PDQ-8: Development and validation of a short-form parkinson's disease questionnaire , 1997 .

[12]  S. Friend,et al.  The mPower study, Parkinson disease mobile data collected using ResearchKit , 2016, Scientific Data.

[13]  Facundo Mémoli,et al.  Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition , 2007, PBG@Eurographics.

[14]  Anirvan Ghosh,et al.  Evaluation of smartphone‐based testing to generate exploratory outcome measures in a phase 1 Parkinson's disease clinical trial , 2018, Movement disorders : official journal of the Movement Disorder Society.

[15]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[16]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[17]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[18]  Andreas W. Kempa-Liehr,et al.  Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh - A Python package) , 2018, Neurocomputing.

[19]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[20]  Joan Cabestany,et al.  Deep learning for freezing of gait detection in Parkinson's disease patients in their homes using a waist-worn inertial measurement unit , 2018, Knowl. Based Syst..

[21]  Larsson Omberg,et al.  Learning Disease vs Participant Signatures: a permutation test approach to detect identity confounding in machine learning diagnostic applications , 2017 .

[22]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[23]  R. Califf,et al.  Real-World Evidence - What Is It and What Can It Tell Us? , 2016, The New England journal of medicine.

[24]  Nir Giladi,et al.  Relationship between freezing of gait (FOG) and other features of Parkinson’s: FOG is not correlated with bradykinesia , 2003, Journal of Clinical Neuroscience.

[25]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[26]  R. Norel,et al.  The self-assessment trap: can we all be better than average? , 2011, Molecular systems biology.

[27]  Witold R. Rudnicki,et al.  Feature Selection with the Boruta Package , 2010 .

[28]  J. Jankovic,et al.  Movement Disorder Society‐sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS‐UPDRS): Scale presentation and clinimetric testing results , 2008, Movement disorders : official journal of the Movement Disorder Society.

[29]  Trevor F. Cox,et al.  Multidimensional Scaling, Second Edition , 2000 .

[30]  Christopher. Simons,et al.  Machine learning with Python , 2017 .