It takes a village: development of digital measures for computer scientists

It takes a village is a phrase often used when people describe the reality of developing digital measures, i.e. technology-based monitoring for clinical trial and healthcare purposes. As digital measures become more complex, in terms of 1) the technologies, methods and data used to derive them, and 2) in terms of the aspects of health that they address, computer scientists, electrical engineers and other people coming from a data or computing background are increasingly important members of this village. To enable more people to bridge the gap from data-orientated background to clinical application of their skills, we present key terms related to digital measures in a unified lexicon to aid communication throughout this process. We highlight key concepts including the critical role of participant engagement in the development of digital measures. We highlight a range of challenges and considerations where computer scientists can have a particular impact on the development of a new digital measure. Presented in accessible language, we build on published best practices, illustrate with concrete examples, and include up-to-date references for further reading.

[1]  I. Clay,et al.  Predicting Subjective Recovery from Lower Limb Surgery Using Consumer Wearables , 2020, Digital Biomarkers.

[2]  P. Natarajan,et al.  Heart rate variability with photoplethysmography in 8 million individuals: a cross-sectional study. , 2020, The Lancet. Digital health.

[3]  Beatrix Vereijken,et al.  A Roadmap to Inform Development, Validation and Approval of Digital Mobility Outcomes: The Mobilise-D Approach , 2020, Digital Biomarkers.

[4]  W. Kibbe,et al.  Investigating sources of inaccuracy in wearable optical heart rate sensors , 2020, npj Digital Medicine.

[5]  P. Bower,et al.  Trials need participants but not their feedback? A scoping review of published papers on the measurement of participant experience of taking part in clinical trials , 2019, Trials.

[6]  Ieuan Clay,et al.  Validity of accelerometry in step detection and gait speed measurement in orthogeriatric patients , 2019, PloS one.

[7]  M. de Vos,et al.  Smartphone-based remote assessment of upper extremity function for multiple sclerosis using the Draw a Shape Test , 2020, Physiological measurement.

[8]  Jennifer C. Goldsack,et al.  Continuous Sound Collection Using Smartphones and Machine Learning to Measure Cough. , 2019, Digital biomarkers.

[9]  Judith H Hibbard,et al.  Patient activation and the use of information to support informed health decisions. , 2017, Patient education and counseling.

[10]  N. Jacobson,et al.  Digital biomarkers of mood disorders and symptom change , 2019, npj Digital Medicine.

[11]  Donald T. Campbell,et al.  Relabeling Internal and External Validity for Applied Social Scientists. , 1986 .

[12]  Dina Katabi,et al.  Passive Monitoring at Home: A Pilot Study in Parkinson Disease , 2019, Digital Biomarkers.

[13]  Beau Woods,et al.  Modernizing and designing evaluation frameworks for connected sensor technologies in medicine , 2020, npj Digital Medicine.

[14]  Suchi Saria,et al.  Using Smartphones and Machine Learning to Quantify Parkinson Disease Severity: The Mobile Parkinson Disease Score , 2018, JAMA neurology.

[15]  C. Berridge,et al.  Sensor-Based Passive Remote Monitoring and Discordant Values: Qualitative Study of the Experiences of Low-Income Immigrant Elders in the United States , 2018, JMIR mHealth and uHealth.

[16]  Ariel V. Dowling,et al.  Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs) , 2020, npj Digital Medicine.

[17]  Elias Bareinboim,et al.  External Validity: From Do-Calculus to Transportability Across Populations , 2014, Probabilistic and Causal Inference.

[18]  J. Goldsack,et al.  Defining and Developing the Workforce Needed for Success in the Digital Era of Medicine , 2020, Digital Biomarkers.

[19]  A. Stern,et al.  Quantifying the use of connected digital products in clinical research , 2020, npj Digital Medicine.

[20]  Ieuan Clay,et al.  The Path Forward for Digital Measures: Suppressing the Desire to Compare Apples and Pineapples , 2020, Digital Biomarkers.

[21]  C. Yancy,et al.  Cardiovascular Health in African Americans: A Scientific Statement From the American Heart Association , 2017, Circulation.

[22]  Hans Ekkehard Plesser,et al.  Reproducibility vs. Replicability: A Brief History of a Confused Terminology , 2018, Front. Neuroinform..

[23]  Matthew B. A. McDermott,et al.  Reproducibility in machine learning for health research: Still a ways to go , 2021, Science Translational Medicine.

[24]  Cyrus K Yamin,et al.  Bridging the digital divide: mobile access to personal health records among patients with diabetes. , 2018, The American journal of managed care.

[25]  Ross D Crosby,et al.  Defining clinically meaningful change in health-related quality of life. , 2003, Journal of clinical epidemiology.

[26]  M. Baker.  1,500 scientists lift the lid on reproducibility , 2016, Nature.

[27]  Maarten De Vos,et al.  Look me in the eye: evaluating the accuracy of smartphone-based eye tracking for potential application in autism spectrum disorder research , 2019, Biomedical engineering online.

[28]  Nick Taptiklis,et al.  Wearable Technology for High-Frequency Cognitive and Mood Assessment in Major Depressive Disorder: Longitudinal Observational Study , 2019, JMIR mental health.

[29]  Jennifer C. Goldsack,et al.  Digital Medicine: A Primer on Measurement , 2019, Digital Biomarkers.

[30]  Amir Muaremi,et al.  Continuous Digital Monitoring of Walking Speed in Frail Elderly Patients: Noninterventional Validation Study and Longitudinal Clinical Trial , 2019, JMIR mHealth and uHealth.

[31]  Akane Sano,et al.  Stress Recognition Using Wearable Sensors and Mobile Phones , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[32]  Ieuan Clay,et al.  Impact of Digital Technologies on Novel Endpoint Capture in Clinical Trials , 2017, Clinical pharmacology and therapeutics.

[33]  S. Friend,et al.  The mPower study, Parkinson disease mobile data collected using ResearchKit , 2016, Scientific Data.

[34]  J. Samet,et al.  Food and Drug Administration , 2007, BMJ : British Medical Journal.

[35]  Hongyu Luo,et al.  Assessment of Fatigue Using Wearable Sensors: A Pilot Study , 2020, Digital Biomarkers.

[36]  T. Hastie,et al.  Wearable sensors enable personalized predictions of clinical laboratory measurements , 2021, Nature Medicine.

[37]  Jennifer C. Goldsack,et al.  Evaluation, Acceptance, and Qualification of Digital Measures: From Proof of Concept to Endpoint , 2021, Digital Biomarkers.

[38]  Dimitrios Athanasiou,et al.  European regulators’ views on a wearable-derived performance measurement of ambulation for Duchenne muscular dystrophy regulatory trials , 2019, Neuromuscular Disorders.

[39]  Christine Manta,et al.  Digital Measures That Matter to Patients: A Framework to Guide the Selection and Development of Digital Measures of Health , 2020, Digital Biomarkers.

[40]  Simon C. Mathews,et al.  A digital health industry cohort across the health continuum , 2020, npj Digital Medicine.

[41]  G. Tison,et al.  Real-world heart rate norms in the Health eHeart study , 2019, npj Digital Medicine.

[42]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[43]  Vera Maljkovic,et al.  Developing Measures of Cognitive Impairment in the Real World from Consumer-Grade Multimodal Sensor Streams , 2019, KDD.

[44]  Peter J Colvonen,et al.  Limiting racial disparities and bias for wearable devices in health science research. , 2020, Sleep.