A User-Centered design and usability testing of a web-based medication reconciliation application integrated in an eHealth network

BACKGROUND Medication discrepancies, which are a threat to patient safety, can be reduced by medication reconciliation (MedRec). MedRec is a complex process that can be supported by the use of information technology and patient engagement. Therefore, the SEAMPAT project aims to develop a MedRec IT platform based on two applications. The application for the professionals is called: the "MedRec app". OBJECTIVE In the present study, we aimed to describe the development and usability testing of the MedRec app, reporting results of a three iterations user-centered usability evaluation. METHODS We used a three phase iterative user-centered study spread over 16 months. At each phase, the usability evaluation included several methods (observations, questionnaires, and follow-up discussions with participants) to collect quantitative and qualitative data in order to improve the current prototype and evolve to the next prototype. RESULTS In total, 48 healthcare professionals (25 general practitioners and 23 hospital clinicians) participated to the MedRec app evaluation. There were 14, 32 and 5 participants for phases 1, 2 and 3 respectively. At each phase, many design modifications were done to strengthen usability. Concerning usability, participants considered the prototypes as an acceptable interface with a median System Usability Score of 73 at phase 2 and 75 at phase 3. Participants emphasized the need for improvements concerning workflow integration, usefulness and interoperability. CONCLUSION The MedRec app was perceived as being useful, usable and satisfying. However, further improvements are required in several usability aspects. Our study demonstrates the importance of conducting usability assessments before investing time and resources in a large study evaluating the effect of an eMedRec approach on clinical outcomes. Our findings may also increase the chances of acceptability and sustained use over time by clinicians.

[1]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[2]  Catherine L. Liang,et al.  Effect of an electronic medication reconciliation application and process redesign on potential adverse drug events: a cluster-randomized trial. , 2009, Archives of internal medicine.

[3]  K. Shojania,et al.  Medication Reconciliation During Transitions of Care as a Patient Safety Strategy , 2013, Annals of Internal Medicine.

[4]  Vimla L. Patel,et al.  Cognitive and usability engineering methods for the evaluation of clinical information systems , 2004, J. Biomed. Informatics.

[5]  Michelle A. Jahn,et al.  Usability evaluation of a medication reconciliation tool: Embedding safety probes to assess users' detection of medication discrepancies , 2018, J. Biomed. Informatics.

[6]  Blake J. Lesselroth,et al.  Medication review software to improve the accuracy of outpatient medication histories: protocol for a randomized controlled trial , 2012 .

[7]  James L. Rudolph,et al.  Characterizing medication discrepancies among older adults during transitions of care: a systematic review focusing on discrepancy synonyms, data sources and classification terms , 2013 .

[8]  David W. Bates,et al.  Development of a tool within the electronic medical record to facilitate medication reconciliation after hospital discharge , 2011, J. Am. Medical Informatics Assoc..

[9]  Rosemary Burke,et al.  Pharmacist-led admission medication reconciliation before and after the implementation of an electronic medication management system , 2017, Int. J. Medical Informatics.

[10]  Tosha B. Wetterneck,et al.  A toolkit to disseminate best practices in inpatient medication reconciliation: multi-center medication reconciliation quality improvement study (MARQUIS). , 2013, Joint Commission journal on quality and patient safety.

[11]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .

[12]  Carol M. Barnum.  Usability Testing Essentials: Ready, Set...Test! , 2010 .

[13]  Diane Gromala,et al.  Usability testing of ANSWER: a web-based methotrexate decision aid for patients with rheumatoid arthritis , 2013, BMC Medical Informatics and Decision Making.

[14]  Anne Spinewine,et al.  Electronic tools to support medication reconciliation: a systematic review , 2017, J. Am. Medical Informatics Assoc..

[15]  Sjaak Brinkkemper,et al.  Computerized Decision Support Improves Medication Review Effectiveness: An Experiment Evaluating the STRIP Assistant’s Usability , 2015, Drugs & Aging.

[16]  Julie N. Babione,et al.  Translating research into practice through user-centered design: An application for osteoarthritis healthcare planning , 2017, Int. J. Medical Informatics.

[17]  Alain Venot,et al.  Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[18]  Andrew J. McLachlan,et al.  Impact of electronic medication reconciliation interventions on medication discrepancies at hospital transitions: a systematic review and meta-analysis , 2016, BMC Medical Informatics and Decision Making.

[19]  Jeff Sauro,et al.  When designing usability questionnaires, does it hurt to be positive? , 2011, CHI.

[20]  Blake J. Lesselroth,et al.  Primary care provider perceptions and use of a novel medication reconciliation technology. , 2011, Informatics in primary care.

[21]  Katie A. Siek,et al.  Colorado Care Tablet: The design of an interoperable Personal Health Application to help older adults with multimorbidity manage their medications , 2010, J. Biomed. Informatics.

[22]  Catherine A. Sugar,et al.  User-centered design to improve clinical decision support in primary care , 2017, Int. J. Medical Informatics.

[23]  Dale Guenter,et al.  A clinical decision support system for chronic pain management in primary care: usability testing and its relevance , 2015, BMJ Health & Care Informatics.

[24]  Tejal K. Gandhi,et al.  Implementation Brief: Evaluation of an Inpatient Computerized Medication Reconciliation System , 2008, J. Am. Medical Informatics Assoc..

[25]  Dillon Chrimes,et al.  Usability testing of Avoiding Diabetes Thru Action Plan Targeting (ADAPT) decision support for integrating care-based counseling of pre-diabetes in an electronic health record , 2014, Int. J. Medical Informatics.

[26]  Ben Shneiderman,et al.  Twinlist: Novel User Interface Designs for Medication Reconciliation , 2013, AMIA.

[27]  An evaluation of medication reconciliation at an outpatient Internal Medicines clinic. , 2017, European journal of internal medicine.

[28]  Emily R. Mackler,et al.  Evaluation of outpatient medication reconciliation involving student pharmacists at a comprehensive cancer center. , 2015, Journal of the American Pharmacists Association : JAPhA.

[29]  Juan Pablo Hourcade,et al.  Personal health records: a randomized trial of effects on elder medication safety , 2014, J. Am. Medical Informatics Assoc..

[30]  Elizabeth M Borycki,et al.  Low-cost rapid usability engineering: designing and customizing usable healthcare information systems. , 2006, Healthcare quarterly.

[31]  Ben Shneiderman,et al.  Medication Reconciliation: Work Domain Ontology, prototype development, and a predictive model. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[32]  Wyeth W. Wasserman,et al.  Usability study of clinical exome analysis software: Top lessons learned and recommendations , 2014, J. Biomed. Informatics.

[33]  Blake J. Lesselroth,et al.  Using consumer-based kiosk technology to improve and standardize medication reconciliation in a specialty care setting. , 2009, Joint Commission journal on quality and patient safety.

[34]  Blackford Middleton,et al.  Design and implementation of a web-based patient portal linked to an electronic health record designed to improve medication safety: the Patient Gateway medications module. , 2008, Informatics in primary care.

[35]  S. Tulledge-Scheitel,et al.  Medication reconciliation in ambulatory care: attempts at improvement , 2009, Quality and Safety in Health Care.

[36]  David Peiris,et al.  Development of an integrated e-health tool for people with, or at high risk of, cardiovascular disease: The Consumer Navigation of Electronic Cardiovascular Tools (CONNECT) web application , 2016, Int. J. Medical Informatics.

[37]  David L. Buckeridge,et al.  Improving patient safety and efficiency of medication reconciliation through the development and adoption of a computer-assisted tool with automated electronic integration of population-based community drug data: the RightRx project , 2017, J. Am. Medical Informatics Assoc..

[38]  J. Schnipper,et al.  The problem with medication reconciliation , 2016, BMJ Quality & Safety.

[39]  Alissa L. Russ,et al.  Design of a Medication Reconciliation Application , 2013, Applied Clinical Informatics.

[40]  George Hripcsak,et al.  Impact of electronic medication reconciliation at hospital admission on clinician workflow. , 2010, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[41]  Hsiu-Fang Hsieh,et al.  Three Approaches to Qualitative Content Analysis , 2005, Qualitative health research.

[42]  Charles Safran,et al.  Medication Harmony: A Framework to Save Time, Improve Accuracy and Increase Patient Activation , 2016, AMIA.

[43]  Tejal K. Gandhi,et al.  Design and implementation of an application and associated services to support interdisciplinary medication reconciliation efforts at an integrated healthcare delivery network. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[44]  Paul Smith,et al.  "Think aloud" and "Near live" usability testing of two complex clinical decision support tools , 2017, Int. J. Medical Informatics.

[45]  Blake J. Lesselroth,et al.  Implementation Brief: Design and Implementation of a Medication Reconciliation Kiosk: the Automated Patient History Intake Device (APHID) , 2009, J. Am. Medical Informatics Assoc..

[46]  Jeff Sauro,et al.  The Factor Structure of the System Usability Scale , 2009, HCI.

[47]  Norman K. Sondheimer,et al.  How Patients Can Improve the Accuracy of their Medical Records , 2014, EGEMS.

[48]  Marco Aiello,et al.  Usability Evaluation of a Web-Based Support System for People With a Schizophrenia Diagnosis , 2012, Journal of medical Internet research.

[49]  Christian M. Rochefort,et al.  Using novel Canadian resources to improve medication reconciliation at discharge: study protocol for a randomized controlled trial , 2012, Trials.

[50]  Henry W. W. Potts,et al.  Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record , 2015, Int. J. Medical Informatics.

[51]  Valéry Ramon,et al.  SEAMPAT - An ICT Platform for Medication Reconciliation with Active Patient Involvement , 2016, ICCHP.

[52]  T. S. Raghu,et al.  Using secure messaging to update medications list in ambulatory care setting , 2015, Int. J. Medical Informatics.

[53]  Tejal K. Gandhi,et al.  Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial , 2012, J. Am. Medical Informatics Assoc..

[54]  Jonathan D. Hron,et al.  Electronic medication reconciliation and medication errors. , 2015, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[55]  Ben Shneiderman,et al.  Novel user interface design for medication reconciliation: an evaluation of Twinlist , 2015, J. Am. Medical Informatics Assoc..

[56]  S. Simon,et al.  Engaging patients in medication reconciliation via a patient portal following hospital discharge. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[57]  A. P. Sherer,et al.  5. Electronic medication reconciliation: A new prescription for the medications matching headache , 2011 .

[58]  Douglas S. Bell,et al.  Potential benefit of electronic pharmacy claims data to prevent medication history errors and resultant inpatient order errors , 2016, J. Am. Medical Informatics Assoc..

[59]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[60]  M. Cantor,et al.  Implementing online medication reconciliation at a large academic medical center. , 2008, Joint Commission journal on quality and patient safety.

[61]  Dean F. Sittig,et al.  Defining health information technology-related errors: new developments since to err is human. , 2011, Archives of internal medicine.

[62]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..