Heuristic evaluation of a telehealth system from the Danish TeleCare North Trial

OBJECTIVE The aim was to evaluate the usability of the design of the telehealth system, named Telekit, developed for the Danish TeleCare North Trial, early into the design process in order to assess potential problems and limitations which could hinder its successful implementation. METHODS Five experts, including one who pilot-tested the Telekit system, individually evaluated its usability and its compliance with Jakob Nielsen's ten usability heuristics for interaction design. Usability problems were categorised according to Rolf Molich's severity classification. RESULTS The five experts identified a total of 152 problems in the Telekit system, each identifying 22-40 problems. 86 (57%) out of the 152 problems were identified only once. All heuristics were used, but the three most frequently used were: "Match between system and the real world" (32%), "Consistency and standards" (13%) and "Aesthetic and minimalist design" (13%). The most widely used classifications were: "Improvement" (40%) and "Minor problem" (43%). CONCLUSION Heuristic evaluation was an effective method for uncovering and identifying problems with the system. The consistent finding of particular usability problems confirms that the development of a telehealth system should pay particular attention to user aspects. The most serious problem was the inability of the system to inform users of how to perform measurements correctly and to "speak the users' language". The problems found in the heuristic evaluation have led to several significant changes in the telehealth system. We suggest that heuristic evaluation always be followed by user tests to evaluate the design of telehealth systems.

[1]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[2]  Jakob Nielsen,et al.  How to Conduct a Heuristic Evaluation , 2006 .

[3]  Elizabeth M Borycki,et al.  Technology induced error and usability: The relationship between usability problems and prescription errors when using a handheld application , 2005, Int. J. Medical Informatics.

[4]  R. Ghiassi,et al.  Health literacy in COPD , 2008, International journal of chronic obstructive pulmonary disease.

[5]  Marie-Catherine Beuscart-Zéphir,et al.  A rapid usability assessment methodology to support the choice of clinical information systems: a case study , 2002, AMIA.

[6]  Vimla L. Patel,et al.  Heuristic evaluation of infusion pumps: implications for patient safety in Intensive Care Units , 2004, Int. J. Medical Informatics.

[7]  O. Hejlesen,et al.  Effectiveness and cost-effectiveness of telehealthcare for chronic obstructive pulmonary disease: study protocol for a cluster randomized controlled trial , 2014, Trials.

[8]  C R Doarn,et al.  Heuristic evaluation of a web-based interface for internet telemedicine. , 1999, Telemedicine journal : the official journal of the American Telemedicine Association.

[9]  Ole K. Hejlesen,et al.  User Satisfaction and Experience with a Telehealth System for the Danish TeleCare North Trial: A Think-Aloud Study , 2014, MIE.

[10]  Nigel Bevan,et al.  International Standards for HCI , 2006 .

[11]  Jakob Nielsen,et al.  Heuristics for User Interface Design , 2006 .

[12]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[13]  Seewon Ryu Telemedicine: Opportunities and Developments in Member States: Report on the Second Global Survey on eHealth 2009 (Global Observatory for eHealth Series, Volume 2) , 2012, Healthcare Informatics Research.

[14]  Linda Little,et al.  E-health , 2008, BCS HCI.

[15]  Marcel Zeelenberg,et al.  Data analysis 2 , 2016 .

[16]  Cristina Masella,et al.  Healthcare continuity from hospital to territory in Lombardy: TELEMACO project. , 2012, The American journal of managed care.

[17]  Jiajie Zhang,et al.  Applying heuristic evaluation to improve the usability of a telemedicine system. , 2006, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[18]  Peter R. Croll,et al.  Investigating risk exposure in e-health systems , 2007, Int. J. Medical Informatics.

[19]  Suzanne Bakken,et al.  Web-based education for low-literate parents in Neonatal Intensive Care Unit: Development of a website and heuristic evaluation and usability testing , 2010, Int. J. Medical Informatics.