Usability Testing

Usability testing is a means for measuring how well people can use some human-made object (such as a web page, a computer interface, a document, or a device) for its intended purpose, i.e. usabili ty testing measures the usability of the object. Usability testing focuses on a particular object or a small set of objects, whereas general human-computer interaction studies attempt to formulate universal principles.

[1]  S. S. Stevens.  Mathematics, measurement, and psychophysics. , 1951 .

[2]  Frederic M. Lord,et al.  On the Statistical Treatment of Football Numbers. , 1953 .

[3]  N. Tallent.  Psychological testing. , 1960, The American journal of nursing.

[4]  H. Robinson.  Principles and Procedures of Statistics , 1961 .

[5]  Seymour Banks,et al.  Experimentation in marketing , 1965 .

[6]  S. O. Parsons,et al.  Human Factors Society , 1966 .

[7]  Rolene B. Cain.  Elementary statistical concepts , 1972 .

[8]  Richard J. Harris.  A primer of multivariate statistics , 1975 .

[9]  D. Massaro.  Experimental psychology and information processing , 1975 .

[10]  James V. Bradley,et al.  Probability, decision, statistics , 1976 .

[11]  Timothy D. Wilson,et al.  Telling more than we can know: Verbal reports on mental processes. , 1977 .

[12]  F. E. Brown.  Marketing research : a structure for decision making , 1980 .

[13]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[14]  W. R. Ford,et al.  Tutorials for the first-time computer user , 1981, IEEE Transactions on Professional Communication.

[15]  Eric Harslem,et al.  Designing the STAR User Interface , 1987, ECICS.

[16]  Peter J. Kennedy.  Development and Testing of the Operator Training Package for a Small Computer System , 1982 .

[17]  James R. Lewis.  Testing Small System Customer Set-Up , 1982 .

[18]  John D. Gould,et al.  Human factors challenges in creating a principal support office system—the speech filing system approach , 1983, TOIS.

[19]  Donald A. Norman,et al.  Design rules based on analyses of human error , 1983, CACM.

[20]  Clayton Lewis,et al.  Designing for usability—key principles and what designers think , 1983, CHI '83.

[21]  J. F. Kelley,et al.  An iterative design methodology for user-friendly natural language office information applications , 1984, TOIS.

[22]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[23]  Pamela L. Alreck,et al.  The Survey Research Handbook , 1984 .

[24]  J. Rassmusen,et al.  Information Processing and Human - Machine Interaction: An Approach to Cognitive Engineering , 1986 .

[25]  Louis M. Gomez,et al.  A Cognitive Analysis of Database Query Production , 1986 .

[26]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[27]  John D. Gould,et al.  The 1984 Olympic Message System: a test of behavioral principles of system design , 1987, CACM.

[28]  John D. Gould,et al.  How to design usable systems , 1995 .

[29]  Donald A. Norman,et al.  Designing for error , 1987 .

[30]  Wendy Gordon,et al.  Qualitative Market Research: A Practitioner's and Buyer's Guide , 1988 .

[31]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[32]  John L. Bennett,et al.  Usability Engineering: Our Experience and Evolution , 1988 .

[33]  A. Chapanis.  Some Generalizations about Generalization , 1988 .

[34]  S. Rosenbaum,et al.  Usability evaluations versus usability testing: when and why? , 1989 .

[35]  Jan H. Spyridakis,et al.  The relevance of reliability and validity to usability testing , 1989 .

[36]  Colin G. Drury,et al.  A Taxonomy of Independent Variables Affecting Human Performance , 1989, Int. J. Man Mach. Stud..

[37]  B. Moffat,et al.  Normalized Performance Ratio - A Measure of the Degree to which a Man-Machine Interface Accomplishes Its Operational Objective , 1990, Int. J. Man Mach. Stud..

[38]  Robert A. Virzi.  Streamlining the Design Process: Running Fewer Subjects , 1990 .

[39]  Joseph B. Sidowski,et al.  Measurements of computer satisfaction, literacy, and aptitudes: A review , 1990, Int. J. Hum. Comput. Interact..

[40]  Stephen Dubin.  How many subjects? Statistical power analysis in research , 1990 .

[41]  D. Broadbent,et al.  The role of instruction and verbalization in improving performance on complex search tasks , 1990 .

[42]  Victoria A. Bowers.  Concurrent versus Retrospective Verbal Protocol for Comparing Window Usability , 1990 .

[43]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[44]  James R. Lewis,et al.  Integrated office software benchmarks: A case study , 1990, INTERACT.

[45]  Chris Marshall,et al.  Usability of product X-lessons from a real product , 1990 .

[46]  Patrick A. Holleran,et al.  A methodological note on pitfalls in usability testing , 1991 .

[47]  Peter C. Wright,et al.  A Cost-Effective Evaluation Method for Use by Designers , 1991, Int. J. Man Mach. Stud..

[48]  James R. Lewis,et al.  A Rank-Based Method for the Usability Comparison of Competing Products , 1991 .

[49]  James R. Lewis,et al.  Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ , 1991, SGCH.

[50]  James Fisher,et al.  Defining the novice user , 1991 .

[51]  Jochen Prümper,et al.  Some surprising differences between novice and expert errors in computerized office work , 1992 .

[52]  Robin Jeffries,et al.  Usability testing vs. heuristic evaluation: was there a contest? , 1992, SGCH.

[53]  Robert W. Bailey,et al.  Usability Testing vs. Heuristic Evaluation: A Head-to-Head Comparison , 1992 .

[54]  David W. Biers,et al.  Team Usability Testing: Are two Heads Better than One? , 1992 .

[55]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[56]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[57]  Richard B. Wright,et al.  Method Bias and Concurrent Verbal Protocol in Software Usability Testing , 1992 .

[58]  James R. Lewis.  Psychometric Evaluation of the Post-Study System Usability Questionnaire: The PSSUQ , 1992 .

[59]  Jakob Nielsen,et al.  A mathematical model of the finding of usability problems , 1993, INTERCHI.

[60]  James R. Lewis,et al.  Multipoint scales: Mean and median differences and observed significance levels , 1993, Int. J. Hum. Comput. Interact..

[61]  Stephen J. Westerman,et al.  Individual differences in human-computer interaction , 1993 .

[62]  Leslie Beth Herbert,et al.  A Comparison of Three Usability Evaluation Methods: Heuristic, Think-Aloud, and Performance Testing , 1993 .

[63]  Michael E. Atwood,et al.  What is gained and lost when using evaluation methods other than empirical testing , 1993 .

[64]  Gregg Skip Bailey,et al.  Iterative methodology and designer training in human-computer interface design , 1993, INTERCHI.

[65]  Richard E. Cordes.  The effects of running fewer subjects on time-on-task measures , 1993, Int. J. Hum. Comput. Interact..

[66]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[67]  J R Lewis,et al.  Sample Sizes for Usability Studies: Additional Considerations , 1994, Human factors.

[68]  J. Nielsen.  Usability inspection methods , 1994, CHI Conference Companion.

[69]  Jakob Nielsen,et al.  Usability laboratories , 1994, Behavior and Information Technology.

[70]  Daniel M. Wildman.  Getting the most from paired-user testing , 1995, INTR.

[71]  Joseph S. Dumas,et al.  Expert Reviews: How Many Experts is Enough? , 1995 .

[72]  R. Abelson.  Statistics As Principled Argument , 1995 .

[73]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[74]  Debora Shaw,et al.  Handbook of usability testing: How to plan, design, and conduct effective tests , 1996 .

[75]  Jurek Kirakowski,et al.  The Software Usability Measurement Inventory: Background and Usage , 1996 .

[76]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[77]  James R. Lewis.  BINOMIAL CONFIDENCE INTERVALS FOR SMALL SAMPLE USABILITY STUDIES , 1996 .

[78]  Miles MacLeod,et al.  The MUSiC performance measurement method , 1997, Behav. Inf. Technol..

[79]  Kay M. Stanney,et al.  Development and Evaluation of the Windows Computer Experience Questionnaire (WCEQ) , 1997, Int. J. Hum. Comput. Interact..

[80]  John Karat,et al.  User-Centered Software Evaluation Methodologies , 1997 .

[81]  Stephanie Rosenbaum,et al.  Usability studies of WWW sites: heuristic evaluation vs. laboratory testing , 1997, SIGDOC '97.

[82]  Clare-Marie Karat,et al.  Cost-Justifying Usability Engineering in the Software Life Cycle , 1997 .

[83]  Thomas K. Landauer,et al.  Behavioral Research Methods in Human-Computer Interaction , 1997 .

[84]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[85]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[86]  Wayne D. Gray,et al.  Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods , 1998, Hum. Comput. Interact..

[87]  J. Krosnick,et al.  Survey research. , 1999, Annual review of psychology.

[88]  Lars Schmidt,et al.  Comparative evaluation of usability tests , 1999, CHI Extended Abstracts.

[89]  Gilbert Cockton,et al.  A Framework for Usability Problem Extraction , 1999, INTERACT.

[90]  R. Jayasuriya,et al.  A review of the construct of computer experience , 1999 .

[91]  A. T. Church,et al.  A Cross-Cultural Study of Response Biases in Personality Measures☆ , 1999 .

[92]  Jean Scholtz,et al.  Common industry format for usability test reports , 2000, CHI Extended Abstracts.

[93]  Marc Hassenzahl.  Prioritizing usability problems: Data-driven and judgement-driven severity estimates , 2000, Behav. Inf. Technol..

[94]  Kyung-Sun Kim,et al.  Cognitive style and on-line database search experience as predictors of Web search performance , 2000, J. Am. Soc. Inf. Sci..

[95]  The design response to usability test findings: a case study based on artifacts and interviews , 2000, SIGDOC.

[96]  H. Rex Hartson,et al.  Testing a Framework for Reliable Classification of Usability Problems , 2000 .

[97]  Emile L. Morse.  The IUSR project and the common industry reporting format , 2000, CUU '00.

[98]  Ted Boren,et al.  Thinking aloud: reconciling theory and practice , 2000 .

[99]  S. Lilienfeld,et al.  The Scientific Status of Projective Techniques , 2000, Psychological science in the public interest : a journal of the American Psychological Society.

[100]  J. Jackson Barnette,et al.  Effects of Stem and Likert Response Option Reversals on Survey Internal Consistency: If You Feel the Need, There is a Better Alternative to Using those Negatively Worded Stems , 2000 .

[101]  David A. Caulton.  Relaxing the homogeneity assumption in usability testing , 2001, Behav. Inf. Technol..

[102]  Hans Baumgartner,et al.  Response Styles in Marketing Research: A Cross-National Investigation , 2001 .

[103]  Jo Wood,et al.  On the reliability of usability testing , 2001, CHI Extended Abstracts.

[104]  Richard E. Cordes,et al.  Task-Selection Bias: A Case for User-Defined Tasks , 2001, Int. J. Hum. Comput. Interact..

[105]  James R. Lewis.  Introduction: Current Issues in Usability Evaluation , 2001, Int. J. Hum. Comput. Interact..

[106]  K. Leung,et al.  Personality in cultural context: methodological issues. , 2001, Journal of personality.

[107]  James R. Lewis,et al.  Evaluation of Procedures for Adjusting Problem-Discovery Rates Estimated From Small Samples , 2001, Int. J. Hum. Comput. Interact..

[108]  Gilbert Cockton,et al.  Why and when five test users aren’t enough , 2001 .

[109]  A. M. Ibrahim,et al.  Differential Responding to Positive and Negative Items: The Case of a Negative Item in a Questionnaire for Course and Faculty Evaluation , 2001, Psychological reports.

[110]  Irvine Clarke,et al.  Extreme response style in cross‐cultural research , 2001 .

[111]  Frank E. Ritter,et al.  Using Multidisciplinary Expert Evaluations to Test and Improve Cognitive Model Interfaces , 2002 .

[112]  Gavriel Salvendy,et al.  Effectiveness of user testing and heuristic evaluation as a function of performance classification , 2002, Behav. Inf. Technol..

[113]  Claude J. Elie,et al.  Remote Usability Evaluation: Overview and Case Studies , 2002, Int. J. Hum. Comput. Interact..

[114]  Joseph S. Dumas,et al.  User-based evaluations , 2002 .

[115]  James R. Lewis,et al.  Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies , 2002, Int. J. Hum. Comput. Interact..

[116]  Karel Vredenburg,et al.  A survey of user-centered design practice , 2002, CHI.

[117]  Jim Rutherford,et al.  Practical Experiment Designs for Engineers and Scientists , 2002, Technometrics.

[118]  L. Faulkner.  Beyond the five-user assumption: Benefits of increased sample sizes in usability testing , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[119]  Morten Hertzum,et al.  The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[120]  Dennis R. Wixon.  Evaluating usability methods: why the current literature fails the practitioner , 2003, INTR.