Model-based evaluation

This chapter will first argue that model-based evaluation is a valuable supplement to conventional usability evaluation, and then survey the current approaches for performing model-based evaluation. Because of the considerable technical detail involved in applying model-based evaluation techniques, this chapter cannot include “how to” guides on the specific modeling methods, but they are all well documented elsewhere. Instead, this chapter will present several high-level issues in constructing and using models for interface evaluation, and comment on the current approaches in the context of those issues. This will assist the reader in deciding whether to apply a model-based technique, which one to use, what problems to avoid, and what benefits to expect. Somewhat more detail will be presented about one form of model-based evaluation, GOMS models, which is a well-developed, relatively simple and “ready to use” methodology applicable to many interface design problems. A set of concluding recommendations will summarize the practical advice.

[1]  Mark O. Riedl,et al.  A perception/action substrate for cognitive modeling in HCI , 2001, Int. J. Hum. Comput. Stud..

[2]  David E. Kieras,et al.  Extending goms to human error and applying it to error-tolerant design , 2000 .

[3]  A. Newell Unified Theories of Cognition , 1990 .

[4]  Norman E. Lane,et al.  THE HUMAN OPERATOR SIMULATOR: AN OVERVIEW , 1981 .

[5]  K. Ronald Laughery Micro SAINT — A Tool for Modeling Human Performance in Systems , 1989 .

[6]  Thomas K. Landauer,et al.  Trouble with Computers: Usefulness, Usability, and Productivity , 1996 .

[7]  Wayne D. Gray,et al.  Milliseconds Matter: an Introduction to Microstrategies and to Their Use in Describing and Predicting Interactive Behavior Milliseconds Matter: an Introduction to Microstrategies and to Their Use in Describing and Predicting Interactive Behavior , 2022 .

[8]  Beverly M. Huey,et al.  Human Performance Models for Computer-Aided Engineering , 1990 .

[9]  John R. Anderson The Architecture of Cognition , 1983 .

[10]  Keith Duncan,et al.  Cognitive Engineering , 2017, Encyclopedia of GIS.

[11]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[12]  David E. Kieras,et al.  Task Analysis and the Design of Functionality , 2014, The Computer Science and Engineering Handbook.

[13]  Richard W. Pew,et al.  Critical Review and Analysis of Performance Models Applicable to Man-Machine Systems Evaluation. , 1977 .

[14]  David E. Kieras,et al.  An Approach to the Formal Analysis of User Complexity , 1999, Int. J. Man Mach. Stud..

[15]  David E. Kieras,et al.  A Guide to GOMS Model Usability Evaluation using NGOMSL , 1997 .

[16]  Henry Petroski,et al.  To Engineer Is Human: The Role of Failure in Successful Design , 1986 .

[17]  F. A. Glenn,et al.  Human Operator Simulation in the Cognitive Domain , 1982 .

[18]  Michael Freed,et al.  Automating CPM-GOMS , 2002, CHI.

[19]  Bonnie E. John,et al.  Exploring the unrealized potential of computer-aided drafting , 1996, CHI.

[20]  David E. Kieras,et al.  Predictive engineering models based on the EPIC architecture for a multimodal high-performance human-computer interaction task , 1997, TCHI.

[21]  Michael E. Atwood,et al.  Project Ernestine: Validating a GOMS Analysis for Predicting and Explaining Real-World Task Performance , 1993, Hum. Comput. Interact..

[22]  Gerald P. Chubb SAINT, A Digital Simulation Language for the Study of Manned Systems , 1981 .

[23]  David E. Kieras,et al.  Towards demystification of direct manipulation: cognitive modeling charts the gulf of execution , 2001, CHI.

[24]  Dennis Wixon,et al.  User performance with command, menu, and iconic interfaces , 1985, CHI '85.

[25]  David E. Kieras,et al.  Towards a Practical GOMS Model Methodology for User Interface Design , 1988 .

[26]  David E. Kieras,et al.  Using GOMS for user interface design and evaluation: which technique? , 1996, TCHI.

[27]  Allen Newell,et al.  The keystroke-level model for user performance time with interactive systems , 1980, CACM.

[28]  Robert Sutton,et al.  Applications of Human Performance Models to System Design , 1989, Springer US.

[29]  A. O. Dick,et al.  The Human Operator Simulator (HOS-IV) , 1989 .

[30]  Regina M. Harris,et al.  Everything you always Wanted to Know about HOS Micromodels but were Afraid to Ask , 1988 .

[31]  David E. Kieras,et al.  The Acquisition and Performance of Text-Editing Skill: A Cognitive Complexity Analysis , 1990, Hum. Comput. Interact..

[32]  Joan M. Ryder,et al.  Developing a Multi-Tasking Cognitive Agent Using the COGNET/iGEN Integrative Architecture , 2000 .

[33]  David E. Kieras,et al.  The GOMS family of user interface analysis techniques: comparison and contrast , 1996, TCHI.

[34]  J. H. Larkin,et al.  Display-based problem solving , 1989 .

[35]  Robert J. Wherry The Human Operator Simulator — HOS , 1976 .

[36]  Anthony J. Hornof,et al.  GLEAN: a computer-based tool for rapid GOMS model usability evaluation of user interface designs , 1995, UIST '95.

[37]  Michael D. Byrne,et al.  A comparison of tools for building GOMS models , 2000, CHI.