Usability testing of two e-learning resources: methods to maximize potential for clinician use.

RATIONALE AND OBJECTIVES Rigorous usability testing of e-learn-ing resources is an important prerequisite to their wide-spread use among clinicians. This study demonstrates the application of an evidence-based approach to usability testing of two stroke-related e-learning resources (StrokEngine). METHODS 14 stroke rehabilitation clinicians (occupational therapists and physiotherapists) from Ontario, Canada participated in a 1.5 h in-person testing session. Clinicians navigated StrokEngine in search of information to answer questions on stroke assessment/intervention. Their search patterns were observed and clinicians provided verbal/written feedback about StrokEngine. Content analysis was used to generate themes and categorize them under two broad categories: facilitators and barriers to use. RESULTS Five key facilitators and three key barriers to Strok-Engine use were identified and related to screen format, layout/organization, ease of navigation, quality of content, likelihood of using StrokEngine in the future, and system dysfunctions. All 14 clinicians were very or extremely satisfied with the layout/organization, quality and clinical relevance of the content, stating that they were likely to use StrokEngine in the future. CONCLUSION All identified barriers from this study were addressed with website modifications in order to maximize the usability and navigability of StrokEngine. This rigorous methodology for usability testing can be applied during the design process of any e-learning resource.

[1]  M. Verrier,et al.  The Community Balance and Mobility Scale-a balance measure for individuals with traumatic brain injury , 2006, Clinical rehabilitation.

[2]  B. Glaser.  The Constant Comparative Method of Qualitative Analysis , 1965 .

[3]  J. Langston,et al.  The Neurobehavioral Cognitive Status Examination: a brief but quantitative approach to cognitive assessment. , 1987, Annals of internal medicine.

[4]  Lorraine N. Smith,et al.  Stroke education for healthcare professionals: making it fit for purpose. , 2008, Nurse education today.

[5]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[6]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[7]  N. Korner-Bitensky,et al.  Actual vs best practice for families post-stroke according to three rehabilitation disciplines. , 2007, Journal of rehabilitation medicine.

[8]  Vimla L. Patel,et al.  Cognitive and usability engineering methods for the evaluation of clinical information systems , 2004, J. Biomed. Informatics.

[9]  Monique W. M. Jaspers,et al.  The think aloud method: a guide to user interface design , 2004, Int. J. Medical Informatics.

[10]  P. Mayring.  Qualitative Content Analysis , 2000 .

[11]  Jakob Nielsen,et al.  Estimating the number of subjects needed for a thinking aloud test , 1994, Int. J. Hum. Comput. Stud..

[12]  D. Jette,et al.  Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. , 2003, Physical therapy.

[13]  Sidney Fels,et al.  A framework for evaluating usability of clinical monitoring technology , 2007, Journal of Clinical Monitoring and Computing.

[14]  B. Uzzell,et al.  Clinical neuropsychology of intervention , 1986 .

[15]  Annie McCluskey,et al.  Occupational therapists report a low level of knowledge, skill and involvement in evidence‐based practice , 2003 .

[16]  V. Patel,et al.  Assessment of a computerized patient record system: a cognitive approach to evaluating medical technology. , 1996, M.D. computing : computers in medical practice.

[17]  Andrew Sears and Julie A. Jacko.  The human-computer interaction handbook , 2013 .

[18]  Monika Kastner,et al.  Usability evaluation of a clinical decision support tool for osteoporosis disease management , 2010, Implementation science : IS.

[19]  Joseph S. Dumas,et al.  User-based evaluations , 2002 .

[20]  André Kushniruk,et al.  Evaluation in the design of health information systems: application of approaches emerging from usability engineering , 2002, Comput. Biol. Medicine.

[21]  Theodoros N. Arvanitis,et al.  Harmonising Evidence-based medicine teaching: a study of the outcomes of e-learning in five European countries , 2008, BMC medical education.

[22]  James Davis,et al.  Computer-based teaching is as good as face to face lecture-based teaching of evidence based medicine: a randomized controlled trial , 2008, Medical teacher.

[23]  K. Khan,et al.  Bmc Medical Education Computer-based Teaching Is as Good as Face to Face Lecture-based Teaching of Evidence Based Medicine: a Randomised Controlled Trial , 2007 .

[24]  Lorie A. Kloda,et al.  Creation and pilot testing of StrokEngine: a stroke rehabilitation intervention website for clinicians and families. , 2008, Journal of rehabilitation medicine.

[25]  Competency Assessment in Clinical Populations: An Introduction to the Cognitive Competency Test , 1986 .

[26]  C. Tannenbaum,et al.  Urinary Incontinence After Stroke: Identification, Assessment, and Intervention by Rehabilitation Professionals in Canada , 2007, Stroke.

[27]  N. Korner-Bitensky,et al.  A national survey of occupational therapists' practices related to participation post-stroke. , 2008, Journal of rehabilitation medicine.

[28]  Jeffrey Rubin,et al.  Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests , 1994 .

[29]  D. Cook,et al.  Internet-based learning in the health professions: a meta-analysis. , 2008, JAMA.

[30]  John Sandars,et al.  Twelve Tips on usability testing to develop effective e-learning in medical education , 2010, Medical teacher.

[31]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[32]  Angeliki Poulymenakou,et al.  Developing a Usability Evaluation Method for e-Learning Applications: Beyond Functional Usability , 2009, Int. J. Hum. Comput. Interact..

[33]  Vimla L. Patel,et al.  Usability testing in medical informatics: cognitive approaches to evaluation of information systems and user interfaces , 1997, AMIA.

[34]  Vimla L. Patel,et al.  Research Paper: Impact of a Computer-based Patient Record System on Data Collection, Knowledge Organization, and Reasoning , 2000, J. Am. Medical Informatics Assoc..

[35]  Jenny Ziviani,et al.  Perceptions of evidence‐based practice: A survey of Australian occupational therapists , 2003 .