Getting the right design and the design right

We present a study comparing usability testing of a single interface versus three functionally equivalent but stylistically distinct designs. We found that when presented with a single design, users give significantly higher ratings and were more reluctant to criticize than when presented with the same design in a group of three. Our results imply that by presenting users with alternative design solutions, subjective ratings are less prone to inflation and give rise to more and stronger criticisms when appropriate. Contrary to our expectations, our results also suggest that usability testing by itself, even when multiple designs are presented, is not an effective vehicle for soliciting constructive suggestions about how to improve the design from end users. It is a means to identify problems, not provide solutions.

[1]  Kasper Hornbæk,et al.  Comparing usability problems and redesign proposals as input to practical systems development , 2005, CHI.

[2]  Thomas Erickson,et al.  Notes on design practice: stories and prototypes as catalysts for communication , 1995 .

[3]  David W. Biers,et al.  Usability Evaluation and Prototype Fidelity: Users and Usability Professionals , 1998 .

[4]  Joseph S. Dumas,et al.  Does the Fidelity of Software Prototypes Affect the Perception of Usability? , 1992 .

[5]  Scott Jenson,et al.  The Simplicity Shift: Innovative Design Tactics in a Corporate World , 2002 .

[6]  Jakob Nielsen,et al.  Diversified parallel design: contrasting design approaches , 1994, CHI Conference Companion.

[7]  A. Lancaster,et al.  Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces , 2004, IEEE Transactions on Professional Communication.

[8]  John M. Carroll,et al.  Scenario-based design: envisioning work and technology in system development: john wiley & sons , 1995 .

[9]  Marc Rettig,et al.  Prototyping for tiny fingers , 1994, CACM.

[10]  James A. Landay,et al.  High-Fidelity or Low-Fidelity, Paper or Computer? Choosing Attributes when Testing Web Prototypes , 2002 .

[11]  Carolyn Snyder,et al.  Paper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces , 2003 .

[12]  R. Stanley Dicks Mis-usability: on the uses and misuses of usability testing , 2002, SIGDOC '02.

[13]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[14]  Kari-Jouko Räihä,et al.  Parallel design in the classroom , 1995, CHI '95.

[15]  Manfred Tscheligi,et al.  Paper prototyping - what is it good for?: a comparison of paper- and computer-based low-fidelity prototyping , 2003, CHI Extended Abstracts.

[16]  Jakob Nielsen,et al.  Comparative design review: an exercise in parallel design , 1993, INTERCHI.

[17]  Kenneth R. Stern,et al.  Low vs. high-fidelity prototyping debate , 1996, INTR.

[18]  John Bowers,et al.  Talking through design: requirements and resistance in cooperative prototyping , 1994, CHI '94.