Usability of Mobile Applications Supporting Training in Diagnostic Decision-Making by Radiologists

The objective of this study is to systematically review the usability of mobile applications currently available in radiology to support training in diagnostic decision-making. Two online stores with major market share (Google Play and iTunes) were searched. A multi-step review process was utilized by three usability investigators and five radiology experts to identify eligible applications and extract usability reviews. From 381 applications that were initially identified, user reviews of final 52 applications revealed 79 usability issues. Usability issues were categorized according to Nielsen’s heuristic usability evaluation principles (HE). The top three most frequent types of usability issues were: Naturalness (43), Simplicity (43), and Efficient Interactions (21). Examples of the most frequent usability issues were: lack of information, lack of labeling, and details about images. This study demonstrates the urgent need of usability test to provide evidence-based guidelines to help choose mobile applications that will yield educational and clinical benefits.

[1]  Timo Partala,et al.  The combined walkthrough: measuring behavioral, affective, and cognitive information in usability testing , 2009 .

[2]  John G. Scott,et al.  Delivery of Clinical Preventive Services in Family Medicine Offices , 2005, The Annals of Family Medicine.

[3]  M Uder,et al.  iPads in Breast Imaging - A Phantom Study. , 2014, Geburtshilfe und Frauenheilkunde.

[4]  Karen Renaud,et al.  Predicting technology acceptance and adoption by the elderly: a qualitative study , 2008, SAICSIT '08.

[5]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[6]  Y. Han,et al.  Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System , 2005, Pediatrics.

[7]  J. Wolf,et al.  Diagnostic inaccuracy of smartphone applications for melanoma detection. , 2013, JAMA dermatology.

[8]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[9]  P C Tang,et al.  Major issues in user interface design for health professional workstations: summary and recommendations. , 1994, International journal of bio-medical computing.

[10]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[11]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[12]  Michael G Evanoff,et al.  Flexible image evaluation: iPad versus secondary-class monitors for review of MR spinal emergency cases, a comparative study. , 2012, Academic radiology.

[13]  L. Faulkner.  Beyond the five-user assumption: Benefits of increased sample sizes in usability testing , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[14]  Dario M Torre,et al.  Comparing PDA- and paper-based evaluation of the clinical skills of third-year students. , 2011, WMJ : official publication of the State Medical Society of Wisconsin.

[15]  Cathleen Wharton,et al.  Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces , 1992, Int. J. Man Mach. Stud..

[16]  Leilei Xu,et al.  Reliability Analysis of a Smartphone-aided Measurement Method for the Cobb Angle of Scoliosis , 2012, Journal of spinal disorders & techniques.

[17]  J. M. Christian Bastien,et al.  Usability testing: a review of some methodological and technical aspects of the method , 2010, Int. J. Medical Informatics.

[18]  Andrew Georgiou,et al.  Review Paper: The Impact of Mobile Handheld Technology on Hospital Physicians' Work Practices and Patient Care: A Systematic Review , 2009, J. Am. Medical Informatics Assoc..

[19]  Todd R. Johnson,et al.  A user-centered framework for redesigning health care interfaces , 2005, J. Biomed. Informatics.

[20]  R. Trelease.  Diffusion of innovations: Smartphones and wireless anatomy learning resources , 2008, Anatomical sciences education.

[21]  Elliot K. Fishman,et al.  The iPad as a mobile device for CT display and interpretation: diagnostic accuracy for identification of pulmonary embolism , 2012, Emergency Radiology.

[22]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[23]  Peter Elkin,et al.  The Human Factors Engineering Approach to Biomedical Informatics Projects: State of the Art, Results, Benefits and Challenges , 2007, Yearbook of Medical Informatics.

[24]  T. L. Lewis,et al.  How to identify, assess and utilise mobile medical applications in clinical practice , 2014, International journal of clinical practice.

[25]  Roland Talanow,et al.  Smartphones, tablets and mobile applications for radiology. , 2013, European journal of radiology.

[26]  Marc Berg,et al.  Viewpoint Paper: Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors , 2003, J. Am. Medical Informatics Assoc..

[27]  Derek Flood,et al.  Usability of mobile applications: literature review and rationale for a new usability model , 2013, Journal of Interaction Science.

[28]  A. Localio,et al.  Role of computerized physician order entry systems in facilitating medication errors. , 2005 .

[29]  Frank Maurer,et al.  User-Centered Design and Agile Methods: A Systematic Review , 2011, 2011 AGILE Conference.

[30]  Evan Steele.  EHR implementation: who benefits, who pays? , 2006, Health management technology.