Methodology III: Empirical Evaluation

Evaluation is an essential part of development. There are several good reasons for carrying out user testing in particular. A successful evaluation requires careful planning. Here we describe the issues that you need to take into account and discuss several effective methods that can be used to collect data. User testing reduces the risk that you will deliver a system to your users that is unusable and is therefore ineffective. We also touch briefly on the need make sure that any evaluation that you carry out is conducted according to appropriate ethical guidelines.

[1]  R. LaPiere.  Attitudes vs Actions. 1934. , 1934, International journal of epidemiology.

[2]  Z. Todd,et al.  Mixing methods in psychology : the integration of qualitative and quantitative methods in theory and practice , 2004 .

[3]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[4]  Sidney L. Smith,et al.  Application of Guidelines for Designing User Interface Software , 1985 .

[5]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[6]  Harry Hochheiser,et al.  Research Methods for Human-Computer Interaction , 2008 .

[7]  Jakob Nielsen,et al.  Eyetracking Web Usability , 2009 .

[8]  Mike Kuniavsky,et al.  Observing the User Experience: A Practitioner's Guide to User Research (Morgan Kaufmann Series in Interactive Technologies) (The Morgan Kaufmann Series in Interactive Technologies) , 2003 .

[9]  A. Huberman,et al.  Qualitative Data Analysis: A Methods Sourcebook , 1994 .

[10]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[11]  T. Landauer,et al.  Handbook of Human-Computer Interaction , 1997 .

[12]  William J. Ray,et al.  Methods Toward a Science of Behavior and Experience , 1981 .

[13]  Sidney Dekker,et al.  Anticipating the effects of technological change: A new era of dynamics for human factors , 2000 .

[14]  Clifford Nass,et al.  The media equation - how people treat computers, television, and new media like real people and places , 1996 .

[15]  Jakob Nielsen,et al.  Improving a human-computer dialogue , 1990, CACM.

[16]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[17]  Mary Beth Rosson,et al.  Getting around the task-artifact cycle: how to make claims and design by scenario , 1992, TOIS.

[18]  Mike Kuniavsky,et al.  Observing the User Experience, Second Edition: A Practitioner's Guide to User Research , 2012 .

[19]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[20]  P. Hancock,et al.  Human Mental Workload , 1988 .

[21]  Linda S. Lotto.  Qualitative Data Analysis: A Sourcebook of New Methods , 1986 .

[22]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[23]  C. Marlin Brown,et al.  Human-Computer Interface Design Guidelines , 1998 .

[24]  Robert C. Calfee.  Experimental Methods in Psychology , 1985 .

[25]  Frank E. Ritter,et al.  Running Behavioral Studies With Human Participants: A Practical Guide , 2012 .

[26]  Marc Berg,et al.  Rationalizing Medical Work: Decision-support Techniques and Medical Practices , 2022 .

[27]  Tom Dayton,et al.  Participatory Practices in the Software Lifecycle , 1997 .

[28]  Harvey S. Smallman,et al.  Naive Realism: Misplaced Faith in Realistic Displays , 2005 .

[29]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[30]  John M. Carroll,et al.  Making Use: Scenario-Based Design of Human-Computer Interactions , 2000 .

[31]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[32]  Dennis Wixon,et al.  User performance with command, menu, and iconic interfaces , 1985, CHI '85.

[33]  Urmila Kukreja,et al.  RUI: Recording user input from interfaces under Windows and Mac OS X , 2006, Behavior research methods.

[34]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[35]  Joseph Psotka,et al.  Intelligent tutoring systems : lessons learned , 1988 .

[36]  Harry Budi Santoso,et al.  Measuring the user experience , 2008 .

[37]  D. C. Howell.  Statistical Methods for Psychology , 1987 .

[38]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .