Analyzing and Visualizing Log Files: A Computational Science of Usability

Researchers in human-computer interactions know that software can easily be instrumented to create a trace of user events2 in the interface (which we call a log file) for later analysis. Using these data for studying usability or other HCI questions is certainly attractive. • The data are cheap – data gathering is totally automated. • The data are gathered discretely, so the possibility of a Hawthorne effect is diminished. • Data can be collected outside of the laboratory, while the user is engaged in real tasks, so the results have greater ecological validity. The problem has always been what to do with the volume of data that gets generated. With so much data, it's difficult to determine what's useful and how to display the useful portion in a meaningful way.

[1]  Michel C. Desmarais,et al.  USER KNOWLEDGE EVALUATION: AN EXPERIMENT WITH UNIX™† , 1987 .

[2]  Robert E. Kraut,et al.  Interface design and multivariate analysis of UNIX command use , 1984, TOIS.

[3]  Mark Guzdial,et al.  Deriving Software Usage Patterns from Log Files , 1993 .

[4]  David E. Kieras,et al.  Towards a Practical GOMS Model Methodology for User Interface Design , 1988 .

[5]  Allen Cypher,et al.  The Structure of Users’ Activities , 1986 .

[6]  Valerie J. Shute,et al.  A Large-Scale Evaluation of an Intelligent Discovery World: Smithtown , 1990, Interact. Learn. Environ..

[7]  Kent P. Vaubel,et al.  Inferring User Expertise for Adaptive Interfaces , 1990, Hum. Comput. Interact..

[8]  R. H. Trigg,et al.  Computer support for transcribing recorded activity , 1989, SGCH.

[9]  Albert N. Badre,et al.  Automatic chunk detection in human-computer interaction , 1994, AVI '94.

[10]  Lynne Anderson-Inman,et al.  The electrotext project: hypertext reading patterns of middle school students , 1994 .

[11]  Michael D. Wilson,et al.  Knowledge-based design of graphical responses , 1993, IUI '93.

[12]  Dan R. Olsen,et al.  Interface usage measurements in a user interface management system , 1988, UIST '88.

[13]  James E. McDonald,et al.  Hypertext perspectives: using pathfinder to build hypertext systems , 1990 .

[14]  Scott E. Hudson,et al.  An Environment to Support User Interface Evaluation Using Synchronized Video and Event Trace Recording , 1993 .

[15]  Bonnie E. John,et al.  Browser-Soar: a computational model of a highly interactive task , 1992, CHI.

[16]  Ken Brodlie,et al.  Scientific visualization: techniques and applications , 1992 .

[17]  Carolanne Fisher,et al.  Exploratory sequential data analysis: traditions, techniques and tools , 1993, SGCH.

[18]  David H. Laidlaw,et al.  The application visualization system: a computational environment for scientific visualization , 1989, IEEE Computer Graphics and Applications.

[19]  Richard W. Hamming,et al.  Numerical Methods for Scientists and Engineers , 1962 .

[20]  H. Pagels,et al.  Dreams of Reason: The Computer and the Rise of the Sciences of Complexity , 1989 .

[21]  Louis M. Gomez,et al.  Formative design evaluation of superbook , 1989, TOIS.

[22]  Laura M. Leventhal,et al.  Empirically-based re-design of a hypertext encyclopedia , 1993, CHI '93.

[23]  Albert N. Badre,et al.  A Knowledge-Based System for Capturing Human-Computer Interaction Events: CHIME , 1991 .

[24]  Mark Guzdial,et al.  Characterizing Process Change Using Log File Data , 1993 .

[25]  Norman Wiseman,et al.  An Introductory Guide to Scientific Visualization , 1992, Springer Berlin Heidelberg.

[26]  Michael Good.  The use of logging data in the design of a new text editor , 1985, CHI '85.