Error by design: methods for predicting device usability

Abstract This paper introduces the idea of predicting ‘designer error’ by evaluating devices using Human Error Identification (HEI) techniques. This is demonstrated using Systematic Human Error Reduction and Prediction Approach (SHERPA) and Task Analysis For Error Identification (TAFEI) to evaluate a vending machine. Appraisal criteria which rely upon user opinion, face validity and utilisation are questioned. Instead a quantitative approach, based upon signal detection theory, is recommended. The performance of people using SHERPA and TAFEI are compared with heuristic judgement and each other. The results of these studies show that both SHERPA and TAFEI are better at predicting errors than the heuristic technique. The performance of SHERPA and TAFEI are comparable, giving some confidence in the use of these approaches. It is suggested that using HEI techniques as part of the design and evaluation process could help to make devices easier to use.

[1]  Graham I. Johnson,et al.  Comparing heuristic, user-centred and checklist-based evaluation approaches , 1995 .

[2]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[3]  N. Stanton,et al.  Is utility in the mind of the beholder? A study of ergonomics methods. , 1998, Applied Ergonomics.

[4]  B Kirwan,et al.  Human error identification in human reliability assessment. Part 2: Detailed comparison of techniques. , 1992, Applied ergonomics.

[5]  A. Chapanis.  The Chapanis Chronicles: 50 Years of Human Factors Research, Education, and Design , 1999 .

[6]  Jop Groeneweg,et al.  Accidents at Sea: Multiple Causes and Impossible Consequences , 1987, Int. J. Man Mach. Stud..

[7]  Jakob Nielsen,et al.  Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared , 1993, INTERCHI.

[8]  R C Coile,et al.  The leading edge , 1985, Hospital forum.

[9]  Neville A. Stanton,et al.  Task analysis for error identification: a methodology for designing error-tolerant consumer products , 1994 .

[10]  Thomas K. Landauer,et al.  The trouble with computers , 1995 .

[11]  Neville A. Stanton.  A systems analysis of consumer p roducts , 1997 .

[12]  Mark S. Young,et al.  What price ergonomics? , 1999, Nature.

[13]  Neville A. Stanton,et al.  Human Factors In Consumer Products , 1997 .

[14]  A VirziRobert.  Refining the test phase of usability evaluation , 1992 .

[15]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[16]  Harold Thimbleby,et al.  Can Humans Think ? , 1991 .

[17]  D. Bartram,et al.  Review of personality assessment instruments (level B) for use in occupational settings , 1995 .

[18]  D. Norman.  The psychology of everyday things , 1990 .

[19]  Thomas C. Ormerod,et al.  Understanding interfaces - a handbook of human-computer dialogue , 1994, Computers and people series.

[20]  H. C. Lee,et al.  Error mode prediction. , 1999, Ergonomics.

[21]  M. Hanson.  Contemporary ergonomics 1998 , 1998 .

[22]  Barry Kirwan,et al.  A Guide to Practical Human Reliability Assessment , 1994 .

[23]  Janice Redish,et al.  Task analysis , 2002 .

[24]  Jeremy Busby,et al.  Error and distributed cognition in design , 2001 .

[25]  S. Stevenage,et al.  Learning to predict human error: issues of acceptability, reliability and validity. , 1998, Ergonomics.

[26]  D. E. Embrey,et al.  SHERPA: A systematic human error reduction and prediction approach , 2013 .

[27]  H Thimbleby,et al.  Can humans think? The Ergonomics Society Lecture 1991. , 1991, Ergonomics.

[28]  Neville A. Stanton,et al.  A SYSTEMS APPROACH TO HUMAN ERROR IDENTIFICATION , 1996 .

[29]  A. T. Welford,et al.  The fundamentals of skill , 1968 .

[30]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[31]  Erik Hollnagel,et al.  Human Reliability Analysis: Context and Control , 1994 .

[32]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[33]  George A. Bekey,et al.  Adaptive Finite-State Models of Manual Control Systems , 1968 .

[34]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[35]  C Baber,et al.  Human error identification techniques applied to public technology: predictions compared with observed use. , 1996, Applied ergonomics.

[36]  L. Aiken.  Psychological testing and assessment , 1976 .

[37]  Neville A. Stanton,et al.  Applying structured methods to Eco-innovation. An evaluation of the Product Ideas Tree diagram , 2001 .

[38]  Mark S. Young,et al.  Guide to Methodology in Ergonomics: Designing for Human Use , 1999 .

[39]  D. Norman.  The psychology of everyday things", Basic Books Inc , 1988 .

[40]  C. Wickens.  Engineering psychology and human performance, 2nd ed. , 1992 .