Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study (Preprint)

BACKGROUND Despite the emergence of app evaluation tools, there remains no well-defined process receptive to diverse local needs, rigorous standards, and current content. The need for such a process to assist in the implementation of app evaluation across all medical fields is evident. Such a process has the potential to increase stakeholder engagement and catalyze interest and engagement with present-day app evaluation models. OBJECTIVE This study aimed to develop and pilot test the Technology Evaluation and Assessment Criteria for Health apps (TEACH-apps). METHODS Tailoring a well-known implementation framework, Replicating Effective Programs, we present a new process to approach the challenges faced in implementing app evaluation tools today. As a culmination of our experience implementing this process and feedback from stakeholders, we present the four-part process to aid the implementation of mobile health technology. This paper outlines the theory, evidence, and initial versions of the process. RESULTS The TEACH-apps process is designed to be broadly usable and widely applicable across all fields of health. The process comprises four parts: (1) preconditions (eg, gathering apps and considering local needs), (2) preimplementation (eg, customizing criteria and offering digital skills training), (3) implementation (eg, evaluating apps and creating educational handouts), and (4) maintenance and evolution (eg, repeating the process every 90 days and updating content). TEACH-apps has been tested internally at our hospital, and there is growing interest in partnering health care facilities to test the system at their sites. CONCLUSIONS This implementation framework introduces a process that equips stakeholders, clinicians, and users with the foundational tools to make informed decisions around app use and increase app evaluation engagement. The application of this process may lead to the selection of more culturally appropriate and clinically relevant tools in health care.

[1]  Neither Agree or Disagree , 2020, Definitions.

[2]  John Torous,et al.  Regulating digital health technologies with transparency: the case for dynamic and multi-stakeholder evaluation , 2019, BMC Medicine.

[3]  Amy N. Cohen,et al.  Mobile Phone and Smartphone Use by People With Serious Mental Illness. , 2019, Psychiatric services.

[4]  Lena Otto,et al.  A design and evaluation framework for digital health interventions , 2019, it Inf. Technol..

[5]  Patricia A. Areán,et al.  By the numbers: ratings and utilization of behavioral health mobile applications , 2019, npj Digital Medicine.

[6]  John Torous,et al.  Deriving a practical framework for the evaluation of health apps. , 2019, The Lancet. Digital health.

[7]  J. Bourgeois,et al.  Digital Technology Adoption in Psychiatric Care: an Overview of the Contemporary Shift from Technology to Opportunity , 2019, Journal of Technology in Behavioral Science.

[8]  John Torous,et al.  Towards a consensus around standards for smartphone apps and digital mental health , 2019, World psychiatry : official journal of the World Psychiatric Association.

[9]  J. Onnela,et al.  Understanding the quality, effectiveness and attributes of top-rated smartphone health apps , 2019, Evidence-Based Mental Health.

[10]  B. Lutz,et al.  Introduction , 2018, Situating Global Art.

[11]  Milou A. Feijt,et al.  Perceived Drivers and Barriers to the Adoption of eMental Health by Psychologists: The Construction of the Levels of Adoption of eMental Health Model , 2018, Journal of medical Internet research.

[12]  J. Torous,et al.  A Hierarchical Framework for Evaluation and Informed Decision Making Regarding Smartphone Apps for Clinical Care. , 2018, Psychiatric services.

[13]  Trisha Greenhalgh,et al.  Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies , 2017, Journal of medical Internet research.

[14]  A. Powell,et al.  Interrater Reliability of mHealth App Rating Measures: Analysis of Top Depression and Smoking Cessation Apps , 2016, JMIR mHealth and uHealth.

[15]  Josip Car,et al.  Unaddressed privacy risks in accredited health and wellness apps: a cross-sectional systematic assessment , 2015, BMC Medicine.

[16]  A. Kilbourne,et al.  Implementing evidence-based interventions in health care: application of the replicating effective programs framework , 2007, Implementation science : IS.