Cognitive Walkthroughs: Understanding the Effect of Task-Description Detail on Evaluator Performance

Inspection-based evaluation techniques are popular because they can be fast, require limited formal training, and can find numerous usability problems. Cognitive walkthroughs are one of the most studied techniques, and as a result, the technique has undergone a series of revisions. One such revision, made to speed the evaluation process and reduce the need for formal training in cognitive psychology, incorporated detailed step-by-step task descriptions to guide the evaluation process. This article reports on the first study that investigated the effects of this change when evaluators were learning to apply the technique. The results indicate that providing detailed task descriptions rather than shorter descriptions, as are often used in usability tests, significantly changes the types of problems found. Evaluators given detailed task descriptions found significantly more problems related to the feedback provided by the system but significantly fewer problems related to difficulties locating the necessary ...

[1]  Jakob Nielsen,et al.  Guerrilla HCI: using discount usability engineering to penetrate the intimidation barrier , 1994 .

[2]  Robin Jeffries,et al.  Applying cognitive walkthroughs to more complex user interfaces: experiences, issues, and recommendations , 1992, CHI.

[3]  Clayton Lewis,et al.  Chapter 30 – Cognitive Walkthroughs , 1997 .

[4]  Andrew Sears,et al.  Heuristic Walkthroughs: Finding the Problems Without the Noise , 1997, Int. J. Hum. Comput. Interact..

[5]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[6]  Robert W. Bailey,et al.  Usability Testing vs. Heuristic Evaluation: A Head-to-Head Comparison , 1992 .

[7]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[8]  Cathleen Wharton,et al.  Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces , 1992, Int. J. Man Mach. Stud..

[9]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[10]  Cathleen Wharton,et al.  The cognitive walkthrough method: a practitioner's guide , 1994 .

[11]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[12]  Joseph S. Dumas,et al.  Expert Reviews: How Many Experts is Enough? , 1995 .

[13]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[14]  Cathleen Wharton,et al.  Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces , 1990, CHI '90.

[15]  Dominique L. Scapin,et al.  Evaluating a user interface with ergonomic criteria , 1995, Int. J. Hum. Comput. Interact..

[16]  Jakob Nielsen,et al.  Enhancing the explanatory power of usability heuristics , 1994, CHI '94.

[17]  John Millar Carroll.  The Nurnberg Funnel: Designing Minimalist Instruction for Practical Computer Skill , 1990 .

[18]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.