User Satisfaction Evaluation of the EHR4CR Query Builder: A Multisite Patient Count Cohort System

The Electronic Health Records for Clinical Research (EHR4CR) project aims to develop services and technology for the leverage reuse of Electronic Health Records with the purpose of improving the efficiency of clinical research processes. A pilot program was implemented to generate evidence of the value of using the EHR4CR platform. The user acceptance of the platform is a key success factor in driving the adoption of the EHR4CR platform; thus, it was decided to evaluate the user satisfaction. In this paper, we present the results of a user satisfaction evaluation for the EHR4CR multisite patient count cohort system. This study examined the ability of testers (n = 22 and n = 16 from 5 countries) to perform three main tasks (around 20 minutes per task), after a 30-minute period of self-training. The System Usability Scale score obtained was 55.83 (SD: 15.37), indicating a moderate user satisfaction. The responses to an additional satisfaction questionnaire were positive about the design of the interface and the required procedure to design a query. Nevertheless, the most complex of the three tasks proposed in this test was rated as difficult, indicating a need to improve the system regarding complicated queries.

[1]  K. Lees,et al.  Feasibility Platform for Stroke Studies: An Online Tool to Improve Eligibility Criteria for Clinical Trials , 2015, Stroke.

[2]  Andrew Muddimer,et al.  The effect of experience on system usability scale ratings , 2012 .

[3]  Jianhua Liu,et al.  Computational challenges and human factors influencing the design and use of clinical research participant eligibility pre-screening tools , 2012, BMC Medical Informatics and Decision Making.

[4]  Joyce A. Mitchell,et al.  Evaluating the informatics for integrating biology and the bedside system for clinical research , 2009 .

[5]  Kenneth A Getz,et al.  Assessing the Impact of Protocol Design Changes on Clinical Trial Performance , 2008, American journal of therapeutics.

[6]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[7]  Peter J. Embi,et al.  Physician Perceptions of an Electronic Health Record-based Clinical Trial Alert System: A Survey of Study Participants , 2005, AMIA.

[8]  Thomas S. Tullis,et al.  A Comparison of Questionnaires for Assessing Website Usability , 2004 .

[9]  Suzana Ahmad,et al.  Measuring usability of educational computer games based on the user success rate , 2011, 2011 International Symposium on Humanities, Science and Engineering Research.

[10]  Kraig Finstad.  The system usability scale and non-native English speakers , 2006 .

[11]  Joseph Finkelstein,et al.  The use and interpretation of quasi-experimental studies in medical informatics. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[12]  Patrick McConnell,et al.  The cancer translational research informatics platform , 2008, BMC Medical Informatics Decis. Mak..

[13]  A Taweel,et al.  An Eligibility Criteria Query Language for Heterogeneous Data Warehouses , 2014, Methods of Information in Medicine.

[14]  R Bache,et al.  Piloting the EHR4CR Feasibility Platform across Europe , 2014, Methods of Information in Medicine.

[15]  Kenneth Getz,et al.  Improving Protocol Design Feasibility to Drive Drug Development Economics and Performance , 2014, International journal of environmental research and public health.

[16]  I. Cockburn,et al.  Price Indexes for Clinical Trial Research: A Feasibility Study , 2013 .

[17]  Richard Bache,et al.  An adaptable architecture for patient cohort identification from diverse data sources. , 2013, Journal of the American Medical Informatics Association : JAMIA.

[18]  Peter J. Embi,et al.  Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study , 2012, J. Am. Medical Informatics Assoc..

[19]  Iain Buchan,et al.  FARSITE: evaluation of an automated trial feasibility assessment and recruitment tool , 2011, Trials.

[20]  M. Bhandari,et al.  How to optimize patient recruitment , 2010 .

[21]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[22]  Iñaki Soto Rey,et al.  Efficiency and effectiveness evaluation of an automated multi-country patient count cohort system , 2015, BMC Medical Research Methodology.

[23]  Christel Daniel-Le Bozec,et al.  Using electronic health records for clinical research: The case of the EHR4CR project , 2015, J. Biomed. Informatics.

[24]  Benjamin Trinczek,et al.  Protocol Feasibility Workflow Using an Automated Multi-country Patient Cohort System , 2014, MIE.

[25]  S. Hussain,et al.  The Electronic Healthcare Record for Clinical Research (EHR4CR) information model and terminology. , 2012, Studies in health technology and informatics.

[26]  Douglas MacFadden,et al.  Application of Information Technology The Shared Health Research Information Network ( SHRINE ) : A Prototype Federated Query Tool for Clinical Data Repositories , 2014 .