A comparative evaluation of heuristic-based usability inspection methods

Given that heuristic evaluation (HE) is a popular evaluation method among practitioners despite criticisms surrounding its performance and reliability, there is a need to improve the method's performance. Several studies have shown HE-Plus, an emerging variant of HE, to outperform HE in both effectiveness and reliability. HE-Plus uses the same set of heuristics as HE; the only difference between these two methods is the 'usability problems profile' element in HE-Plus. This paper reports our attempt to verify the original profile employed in HE-Plus based on usability problem classification in the User Action Framework and an experiment evaluating its outcome by comparing HE with two HE variants using a profile (HE-Plus and HE++) and a control group. Our results confirmed the role of the 'usability problems profiles' on improving the performance and reliability of heuristic evaluation: both HE-Plus and HE++ outperformed HE in terms of effectiveness as well as reliability.

[1]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[2]  Jacqueline Brodie,et al.  Applying user testing data to UEM performance metrics , 2004, CHI EA '04.

[3]  Steven M. Belz,et al.  The user action framework: a reliable foundation for usability engineering support tools , 2001, Int. J. Hum. Comput. Stud..

[4]  Joseph S. Dumas,et al.  Comparative usability evaluation (CUE-4) , 2008, Behav. Inf. Technol..

[5]  Nina Khalayli,et al.  Persona based rapid usability kick-off , 2007, CHI Extended Abstracts.

[6]  Donald A. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[7]  Wendy A. Rogers,et al.  The human factors and ergonomics society perspective , 2005, CHI EA '05.

[8]  Jacqueline Brodie,et al.  Extending the Heuristic Evaluation Method through Contextualisation , 2002 .

[9]  Robert C. Williges,et al.  Criteria For Evaluating Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[10]  Wayne D. Gray,et al.  Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods , 1998, Hum. Comput. Interact..

[11]  Andrew Sears,et al.  Heuristic Walkthroughs: Finding the Problems Without the Noise , 1997, Int. J. Hum. Comput. Interact..

[12]  Frederick G. Conrad,et al.  A heuristic evaluation of a World Wide Web prototype , 1996, INTR.

[13]  Arnold M. Lund.  The Need Fora Standardized Set of Usability Metrics , 1998 .

[14]  Gitte Lindgaard,et al.  Usability testing: what have we overlooked? , 2007, CHI.

[15]  Klaus Kaasgaard,et al.  Comparative usability evaluation , 2004, Behav. Inf. Technol..

[16]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.