Behavioral Research Methods in Human-Computer Interaction

Publisher Summary This chapter discusses the conduct of research to guide the development of more useful and usable computer systems. Experimental research in human-computer interaction involves varying the design or deployment of systems, observing the consequences, and inferring from observations what to do differently. For such research to be effective, it must be owned—instituted, trusted and heeded—by those who control the development of new systems. Thus, managers, marketers, systems engineers, project leaders, and designers as well as human factors specialists are important participants in behavioral human-computer interaction research. This chapter is intended as much for those with backgrounds in computer science, engineering, or management as for human factors researchers and cognitive systems designers. It is argued in this chapter that the special goals and difficulties of human-computer interaction research make it different from most psychological research as well as from traditional computer engineering research. The main goal, the improvement of complex, interacting human-computer systems, requires behavioral research but is not sufficiently served by the standard tools of experimental psychology such as factorial controlled experiments on pre-planned variables. The chapter contains about equal quantities of criticism of inappropriate general research methods, description of valuable methods, and prescription of specific useful techniques.

[1]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[2]  Frederick Mosteller,et al.  Data Analysis and Regression , 1978 .

[3]  R. Tibshirani,et al.  Generalized additive models for medical research , 1986, Statistical methods in medical research.

[4]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[5]  Louis M. Gomez,et al.  A Cognitive Analysis of Database Query Production , 1986 .

[6]  John Seely Brown,et al.  Research that reinvents the corporation , 1991 .

[7]  T. K. Landauer,et al.  Selection from alphabetic and numeric menu trees using a touch screen: breadth, depth, and width , 1985, CHI '85.

[8]  Ma Conway,et al.  Handbook of perception and cognition , 1996 .

[9]  Jakob Nielsen,et al.  A mathematical model of the finding of usability problems , 1993, INTERCHI.

[10]  R. Kirk Experimental Design: Procedures for the Behavioral Sciences , 1970 .

[11]  R. Gnanadesikan,et al.  Probability plotting methods for the analysis of data. , 1968, Biometrika.

[12]  John R. Anderson,et al.  The Transfer of Cognitive Skill , 1989 .

[13]  R. Luce,et al.  Measurement, scaling, and psychophysics. , 1988 .

[14]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[15]  John R. Anderson,et al.  The Transfer of Text-Editing Skill , 1985, Int. J. Man Mach. Stud..

[16]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[17]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[18]  R. Abelson Statistics As Principled Argument , 1995 .

[19]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[20]  George C. Derringer,et al.  The design of experiments: Statistical principles for practical applications. , 1990 .

[21]  S. T. Dumais,et al.  Human factors and behavioral science: Statistical semantics: Analysis of the potential performance of key-word information systems , 1983, The Bell System Technical Journal.

[22]  J. Michell An introduction to the logic of psychological measurement , 1995 .

[23]  J. B. Black,et al.  Facilitating Human-Computer Communication , 1981 .

[24]  Larry Tesler Enlisting user help in software design , 1983, SGCH.

[25]  Dennis R. Wixon,et al.  Building a user-derived interface , 1984, CACM.

[26]  Sidney L. Smith,et al.  Guidelines for Designing User Interface Software , 1986 .

[27]  Mark Horton,et al.  An Introduction to Display Editing with Vi , 1998 .

[28]  J. Grudin,et al.  Designing in the dark: logics that compete with the user , 1986, CHI '86.

[29]  Susan T. Dumais,et al.  The vocabulary problem in human-system communication , 1987, CACM.

[30]  William S. Cleveland,et al.  Visualizing Data , 1993 .

[31]  R. A. Fisher,et al.  Design of Experiments , 1936 .

[32]  Frederick Mosteller,et al.  Understanding robust and exploratory data analysis , 1983 .

[33]  Michael E. Lesk,et al.  Hypertext for the electronic library?: CORE sample results , 1991, HYPERTEXT '91.

[34]  Jakob Nielsen,et al.  Usability engineering at a discount , 1989 .

[35]  Christine L. Borgman,et al.  Why are Online Catalogs Hard to Use? Lessons Learned from Information=Retrieval Studies , 1986 .

[36]  John D. Gould,et al.  The 1984 Olympic Message System: a test of behavioral principles of system design , 1987, CACM.

[37]  P. Laycock,et al.  Optimum Experimental Designs , 1995 .

[38]  Thomas K. Landauer,et al.  What Makes a Difference When? Comments on Grudin and Barnard , 1984 .

[39]  Jonathan Grudin,et al.  The Cognitive Demands of Learning and Representing Command Names for Text Editing , 1984 .

[40]  Thomas K. Landauer,et al.  Natural command names and initial learning: a study of text-editing terms , 1983, CACM.

[41]  Dennis E. Egan,et al.  Individual Differences In Human-Computer Interaction , 1988 .

[42]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[43]  R. Mead,et al.  The Design of Experiments: Statistical Principles for Practical Applications. , 1989 .

[44]  B. J. Winer Statistical Principles in Experimental Design , 1992 .

[45]  Beat Kleiner,et al.  Graphical Methods for Data Analysis , 1983 .

[46]  Thomas P. Moran,et al.  The evaluation of text editors: methodology and empirical results. , 1983, CACM.

[47]  R. Luce,et al.  Four tensions concerning mathematical modeling in psychology. , 1995, Annual review of psychology.

[48]  R. Rosenthal,et al.  Experimenter effects in behavioral research , 1968 .

[49]  David M. Allen Modeling Experimental and Observational Data , 1994 .

[50]  C. Judd,et al.  Data analysis: continuing issues in the everyday analysis of psychological data. , 1995, Annual review of psychology.

[51]  Michael H. Kutner Applied Linear Statistical Models , 1974 .

[52]  Thomas K. Landauer,et al.  Trouble with Computers: Usefulness, Usability, and Productivity , 1996 .

[53]  Bob Rehder,et al.  Looking for Transfer and Interference , 1995 .

[54]  Michael Good The Iterative Design of a New Text Editor , 1985 .

[55]  Christopher J. Rowe,et al.  Scanning in the supermarket: for better or worse? A case study in introducing electronic point of sale , 1990 .

[56]  Jaap Van Brakel,et al.  Foundations of measurement , 1983 .

[57]  J. Tukey Conclusions vs Decisions , 1960 .

[58]  G. Vining,et al.  Data Analysis: A Model-Comparison Approach , 1989 .