Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs)

Digital medicine is an interdisciplinary field, drawing together stakeholders with expertize in engineering, manufacturing, clinical science, data science, biostatistics, regulatory science, ethics, patient advocacy, and healthcare policy, to name a few. Although this diversity is undoubtedly valuable, it can lead to confusion regarding terminology and best practices. There are many instances, as we detail in this paper, where a single term is used by different groups to mean different things, as well as cases where multiple terms are used to describe essentially the same concept. Our intent is to clarify core terminology and best practices for the evaluation of Biometric Monitoring Technologies (BioMeTs), without unnecessarily introducing new terms. We focus on the evaluation of BioMeTs as fit-for-purpose for use in clinical trials. However, our intent is for this framework to be instructional to all users of digital measurement tools, regardless of setting or intended use. We propose and describe a three-component framework intended to provide a foundational evaluation framework for BioMeTs. This framework includes (1) verification, (2) analytical validation, and (3) clinical validation. We aim for this common vocabulary to enable more effective communication and collaboration, generate a common and meaningful evidence base for BioMeTs, and improve the accessibility of the digital medicine field.

[1]  Michael S. Deutsch,et al.  Software verification and validation , 1982 .

[2]  E Versi,et al.  "Gold standard" is an appropriate term. , 1992, BMJ.

[3]  G. Breithardt,et al.  Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. , 1996 .

[4]  Whole Grain Label Statements.  Guidance for Industry and FDA Staff , 2006 .

[5]  U. S. Department of Health and Human Services FDA Cen Research,et al.  Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance , 2006, Health and quality of life outcomes.

[6]  Dorin Panescu.  Medical device development , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Anika Ashok,et al.  Guidance for Industry by U.S. Department of Health and Human Services—Food and Drug Administration—Center for Biologics Evaluation and Research (CBER)—February 1999 , 2009 .

[8]  Yutaka Imai,et al.  European Society of Hypertension International Protocol revision 2010 for the validation of blood pressure measuring devices in adults , 2010, Blood pressure monitoring.

[9]  Marco Donati,et al.  An enhanced estimate of initial contact and final contact instants of time using lower trunk inertial sensor data. , 2012, Gait & posture.

[10]  C. Karpman,et al.  Measuring Gait Speed in the Out-Patient Clinic: Methodology and Feasibility , 2014, Respiratory Care.

[11]  K. Kaufman,et al.  Validity of using tri-axial accelerometers to measure human movement - Part II: Step counts at a wide range of gait velocities. , 2014, Medical engineering & physics.

[12]  Diana Trojaniello,et al.  Accuracy, sensitivity and robustness of five different methods for the estimation of gait temporal parameters using a single inertial sensor mounted on the lower trunk. , 2014, Gait & posture.

[13]  David Giles,et al.  Validity of the Polar V800 heart rate monitor to measure RR intervals at rest , 2015, European Journal of Applied Physiology.

[14]  Wenli Wu,et al.  [Mobile Health: IEEE Standard for Wearable Cuffless Blood Pressure Measuring Devices]. , 2015, Zhongguo yi liao qi xie za zhi = Chinese journal of medical instrumentation.

[15]  Binh Q. Tran,et al.  Optimization of an Accelerometer and Gyroscope-Based Fall Detection Algorithm , 2015, J. Sensors.

[16]  A. Godfrey,et al.  Instrumenting gait with an accelerometer: A system and algorithm examination , 2015, Medical engineering & physics.

[17]  Eric Elenko,et al.  Defining digital medicine , 2015, Nature Biotechnology.

[18]  Parisa Farzanehfar,et al.  An Ambulatory Tremor Score for Parkinson's Disease. , 2016, Journal of Parkinson's disease.

[19]  Giuseppe Enrico Bignardi,et al.  Validation and verification of automated urine particle analysers , 2016, Journal of Clinical Pathology.

[20]  Richard Fries.  Hardware Verification and Validation , 2016 .

[21]  T. Insel.  Digital Phenotyping: Technology for a New Science of Behavior. , 2017, JAMA.

[22]  Christopher Kovalchick,et al.  Can composite digital monitoring biomarkers come of age? A framework for utilization , 2017, Journal of clinical and translational science.

[23]  S. Terry.  An Evidence Framework for Genetic Testing. , 2017, Genetic testing and molecular biomarkers.

[24]  Dermot Phelan,et al.  Accuracy of Wrist-Worn Heart Rate Monitors , 2017, JAMA cardiology.

[25]  Martin J. Landray.  Developing novel endpoints, generated using mobile technology, for use in clinical trials: a clinical trials transformation initiative (CTTI) project , 2017 .

[26]  F TerrySharon.  An Evidence Framework for Genetic Testing. , 2017 .

[27]  Alexandra König,et al.  Objective measurement of gait parameters in healthy and cognitively impaired elderly using the dual-task paradigm , 2017, Aging Clinical and Experimental Research.

[28]  Takahiro Okumura,et al.  Development and Validation of a Novel Cuff-Less Blood Pressure Monitoring Device , 2017, JACC. Basic to translational science.

[29]  Jessilyn Dunn,et al.  Wearables and the medical revolution. , 2018, Personalized medicine.

[30]  Gert Bos,et al.  ISO 13485:2003/2016—Medical Devices—Quality Management Systems—Requirements for Regulatory Purposes , 2018 .

[31]  Raquel Bailón,et al.  Validation of Heart Rate Monitor Polar RS800 for Heart Rate Variability Analysis During Exercise , 2016, Journal of strength and conditioning research.

[32]  Kevin J. Deluzio,et al.  Operator Bias Errors Are Reduced Using Standing Marker Alignment Device for Repeated Visit Studies. , 2018, Journal of biomechanical engineering.

[33]  CTTI RECOMMENDATIONS: DEVELOPING NOVEL ENDPOINTS GENERATED BY MOBILE TECHNOLOGY FOR USE IN CLINICAL TRIALS , 2018 .

[34]  Sandy Weininger,et al.  Regulatory Considerations for Physiological Closed-Loop Controlled Medical Devices Used for Automated Critical Care: Food and Drug Administration Workshop Discussion Topics , 2017, Anesthesia and analgesia.

[35]  José R. Banegas,et al.  Relationship between Clinic and Ambulatory Blood‐Pressure Measurements and Mortality , 2018, The New England journal of medicine.

[36]  Clint Hansen,et al.  Validation of a Lower Back “Wearable”-Based Sit-to-Stand and Stand-to-Sit Algorithm for Patients With Parkinson's Disease and Older Adults in a Home-Like Environment , 2018, Front. Neurol..

[37]  R. Bernier,et al.  Can a heart rate variability biomarker identify the presence of autism spectrum disorder in eight year old children , 2018 .

[38]  Jessilyn Dunn,et al.  Windows Into Human Health Through Wearables Data Analytics. , 2019, Current opinion in biomedical engineering.

[39]  Anshu M. Gupta,et al.  Current Diabetes Technology: Striving for the Artificial Pancreas , 2019, Diagnostics.

[40]  Oliver B. Regele,et al.  Digital biomarkers for Alzheimer’s disease: the mobile/wearable devices opportunity , 2019, npj Digital Medicine.

[41]  Andrea Coravos,et al.  Is health-care data the new blood? , 2019, The Lancet. Digital health.

[42]  Jennifer C. Goldsack,et al.  Digital Medicine: A Primer on Measurement , 2019, Digital Biomarkers.

[43]  Medical devices. Quality management systems. Requirements for regulatory purposes , 2022 .