Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared

Two alternative user interface designs were subjected to user testing to measure user performance in a database query task. User performance was also estimated heuristically in three different ways and by use of formal GOMS modelling. The estimated values for absolute user performance had very high variability, but estimates of the relative advantage of the fastest interface were less variable. Choosing the fastest of the two designs would have a net present value more than 1,000 times the cost of getting the estimates. A software manager would make the correct choice every time in our case study if decisions were based on at least three independent estimates. User testing was 4.9 times as expensive as the cheapest heuristic method but provided better performance estimates.

[1]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[2]  Jakob Nielsen,et al.  Usability inspection methods , 1994, CHI 95 Conference Companion.

[3]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[4]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[5]  Jakob Nielsen,et al.  Usability inspection methods: report on a workshop held at CHI'92, Monterey, CA, May 3–4, 1992 , 1993, SGCH.

[6]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[7]  Jakob Nielsen.  Reliability of severity estimates for usability problems found by heuristic evaluation , 1992, CHI '92.

[8]  Victoria Bellotti,et al.  Implications of Current Design Practice for the Use of HCI Techniques , 1988, BCS HCI.

[9]  Gary M. Olson,et al.  The Growth of Cognitive Modeling in Human-Computer Interaction Since GOMS , 1990, Hum. Comput. Interact..

[10]  Michael E. Atwood,et al.  The precis of Project Ernestine or an overview of a validation of GOMS , 1992, CHI.

[11]  Jakob Nielsen,et al.  The usability engineering life cycle , 1992, Computer.

[12]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.

[13]  Marilyn Tremaine,et al.  Cost/benefit analysis for incorporating human factors in the software lifecycle , 1988, CACM.

[14]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.