Usability evaluation considered harmful (some of the time)

Current practice in Human Computer Interaction as encouraged by educational institutes, academic review processes, and institutions with usability groups advocate usability evaluation as a critical part of every design process. This is for good reason: usability evaluation has a significant role to play when conditions warrant it. Yet evaluation can be ineffective and even harmful if naively done 'by rule' rather than 'by thought'. If done during early stage design, it can mute creative ideas that do not conform to current interface norms. If done to test radical innovations, the many interface issues that would likely arise from an immature technology can quash what could have been an inspired vision. If done to validate an academic prototype, it may incorrectly suggest a design's scientific worthiness rather than offer a meaningful critique of how it would be adopted and used in everyday practice. If done without regard to how cultures adopt technology over time, then today's reluctant reactions by users will forestall tomorrow's eager acceptance. The choice of evaluation methodology - if any - must arise from and be appropriate for the actual problem or research question under consideration.

[1]  W. Keith Edwards,et al.  At Home with Ubiquitous Computing: Seven Challenges , 2001, UbiComp.

[2]  Cerstin Mahlow,et al.  Evolution by Evaluation , 2009, Int. J. Inf. Commun. Technol. Educ..

[3]  Harold W. Thimbleby,et al.  User interface design , 1990, ACM Press Frontier Series.

[4]  Carl Gutwin,et al.  A review of groupware evaluations , 2000, Proceedings IEEE 9th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE 2000).

[5]  John D. Gould,et al.  How to design usable systems , 1995 .

[6]  Louise Barkhuus,et al.  From Mice to Men - 24 Years of Evaluation in CHI , 2007, CHI 2007.

[7]  R. Stanley Dicks.  Mis-usability: on the uses and misuses of usability testing , 2002, SIGDOC '02.

[8]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[9]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[10]  RichardB . Buxton,et al.  ITERATION IN THE DESIGN OF THE HUMAN-COMPUTER INTERFACE 1 , 2003 .

[11]  Gregory D. Abowd,et al.  Human-Computer Interaction (3rd Edition) , 2003 .

[12]  Gilbert Cockton,et al.  Make Evaluation Poverty History , 2007, CHI 2007.

[13]  Bill Buxton,et al.  Sketching User Experiences: Getting the Design Right and the Right Design , 2007 .

[14]  Saul Greenberg.  Embedding a Design Studio Course in a Conventional Computer Science Program , 2007, HCIEd.

[15]  Carl Gutwin,et al.  The mechanics of collaboration: developing low cost usability evaluation methods for shared workspaces , 2000, Proceedings IEEE 9th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE 2000).

[16]  Abigail Sellen,et al.  User sketches: a quick, inexpensive, and effective way to elicit more reflective user feedback , 2006, NordiCHI '06.

[17]  Thomas K. Landauer,et al.  Trouble with Computers: Usefulness, Usability, and Productivity , 1996 .

[18]  Douglas C. Engelbart,et al.  A research center for augmenting human intellect , 1968, AFIPS Fall Joint Computing Conference.

[19]  Edsger W. Dijkstra,et al.  Letters to the editor: go to statement considered harmful , 1968, CACM.

[20]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[21]  P. Kidwell,et al.  The trouble with computers: Usefulness, usability and productivity , 1996, IEEE Annals of the History of Computing.

[22]  Edsger W. Dijkstra,et al.  Go To Statement Considered Harmful , 2022, Software Pioneers.

[23]  Saul Greenberg,et al.  Teaching human computer interaction to programmers , 1996, INTR.

[24]  Adrian Snodgrass,et al.  Interpretation in Architecture: Design as Way of Thinking , 2005 .

[25]  Paula Kotzé,et al.  Creativity and HCI: From Experience to Design in Education, Selected Contributions from HCIEd 2007, The Second International Working Conference of Human-Computer Interaction Educators, March 29-30, 2007, Aveiro, Portugal , 2009, HCIEd.

[26]  Tom Carey,et al.  ACM SIGCHI Curricula for Human-Computer Interaction , 1992 .

[27]  S. Greenberg,et al.  The Weak Science of Human-Computer Interaction , 1998 .

[28]  William W. Gaver,et al.  Design: Cultural probes , 1999, INTR.

[29]  Dan R. Olsen,et al.  Evaluating user interface systems research , 2007, UIST.

[30]  Phoebe Sengers,et al.  The Evolution of Evaluation , 2007, CHI 2007.

[31]  Abigail Sellen,et al.  Getting the right design and the design right , 2006, CHI.

[32]  Williams Ja,et al.  FROM MICE TO MEN. , 1965, Lancet.

[33]  Masaki Suwa,et al.  External Representations Contribute to the Dynamic Construction of Ideas , 2002, Diagrams.

[34]  Donald O. Case,et al.  The Trouble with Computers: Usefulness, Usability, and Productivity, by Thomas K. Landauer , 1996, J. Am. Soc. Inf. Sci..