The Added Usefulness of Process Measures Over Performance Measures in Interface Design

Various studies concerning the application of usability evaluation methods have shown the usefulness of these testing methods. The objective of this study was to evaluate the added advantage of collecting process measures (data used to learn how and why an error occurred) over performance measures (data used to determine what went wrong) in interface design. The most salient finding of this study is that when used in the design/redesign process, performance data improved effectiveness (i.e., reduced the number of errors), whereas process measures data improved efficiency by reducing the time needed to complete tasks. The results from this study, as with any other usability study, should be of particular interest to the software industry, which is constantly trying to reduce development time and resources while improving user-friendly interface design.

[1]  Leslie Beth Herbert,et al.  A Comparison of Three Usability Evaluation Methods: Heuristic, Think-Aloud, and Performance Testing , 1993 .

[2]  Peter C. Wright,et al.  The use of think-aloud evaluation methods in design , 1991, SGCH.

[3]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[4]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.

[5]  Gerhard Deffner,et al.  Understanding Perceived Image Quality: New Applications for Verbal Protocol Methodology , 1994 .

[6]  David A. Wroblewski.  The construction of human-computer interfaces considered as a craft , 1991 .

[7]  Mansour Rahimi,et al.  Concurrent and Retrospective Verbal Protocols in Usability Testing: Is there Value Added in Collecting Both? , 1995 .

[8]  Harald Reiterer,et al.  Evaluation of user interfaces: EVADIS II - a comprehensive evaluation approach , 1993, Behav. Inf. Technol..

[9]  Jane M. Fritz.  HyperCard applications for teaching information systems , 1991, SIGCSE '91.

[10]  Gavriel Salvendy,et al.  A proposed index of usability: A method for comparing the relative usability of different software systems , 1997, Behav. Inf. Technol..

[11]  Tiziana Catarci,et al.  Visual Modeling of Temporal Data in Usability Experiments , 1998, VDB.

[12]  Gerhard P. Deffner,et al.  Evaluation of Concurrent Thinking Aloud Using Eye-tracking Data , 1990 .

[13]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[14]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[15]  Alan E. Benson,et al.  Are We Overlooking Some Usability Testing Methods? A Comparison of Lab, Beta, and Forum Tests , 1993, Behav. Inf. Technol..

[16]  Susan M. Denning,et al.  The Value of Thinking-Aloud Protocols in Industry: A Case Study at Microsoft Corporation , 1990 .

[17]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[18]  John Dowell,et al.  A framework for human factors evaluation , 1991 .

[19]  Laurel Allender,et al.  Why choose? A process approach to usability testing , 1995 .

[20]  Clayton Lewis,et al.  Chapter 30 – Cognitive Walkthroughs , 1997 .