Applying the results of transaction log analysis

The ability to conduct unobtrusive observation of user searching is a potential strength of the method of information retrieval system analysis known as transaction log analysis (TLA). Transaction logs supply unequivocal information about what a user typed while searching. All other methods rely on self‐reporting, which, as Nielsen points out, is not always corroborated by the logs. Regardless of where in an institution information retrieval (IR) system evaluation takes place, TLA is a method that enables library staff at all levels to examine a variety of system and user‐related activities that are recorded on the log. Dominick suggested that TLA can enable the examination of three broad categories of activity: 1) system performance and resource utilization, 2) information retrieval performance, and 3) user interaction with the IR system. This article has been divided into several sections corresponding to functional areas in a library to suggest useful applications of TLA.

[1]  Sally W. Kalin The Searching Behavior of Remote Users: A Study of One Online Public Access Catalog (OPAC). , 1991 .

[2]  Terry Ballard,et al.  The Human Interface: an Ongoing Study of OPAC Usage at Adelphi University , 1992 .

[3]  Neal K. Kaske A comparative study of subject searching in an OPAC among branch libraries of a university library system , 1988 .

[4]  Walt Crawford The RLIN command analysis system: measuring use and performance of an online system , 1985 .

[5]  Christine L. Borgman,et al.  Research report prepared for OCLC on end user behavior on the Ohio state university libraries'online catalog : a computer monitoring study , 1983 .

[6]  Stephen Walker,et al.  Designing an online public access catalogue: Okapi, a catalogue on a Local Area Network , 1985 .

[7]  Christine L. Borgman,et al.  Why are Online Catalogs Hard to Use? Lessons Learned from Information=Retrieval Studies , 1986 .

[8]  Martin Kurth,et al.  Controlled and Uncontrolled Vocabulary Subject Searching in an Academic Library Online Catalog. , 1991 .

[9]  M A Cahan GRATEFUL MED: a tool for studying searching behavior. , 1989, Medical reference services quarterly.

[10]  Arlene G. Taylor,et al.  Authority Files in Online Catalogs , 1984 .

[11]  James G. Schmidt,et al.  Capacity modeling at RLG , 1988 .

[12]  Stephen E. Robertson,et al.  Evaluation of online catalogues: Eliciting information from the user , 1991, Inf. Process. Manag..

[13]  Ray R. Larson The decline of subject searching: long-term trends and patterns of index use in an online catalog , 1991 .

[14]  Roger W. Elliott Kiviat-graphs as a means for displaying performance data for on-line retrieval systems , 1976, J. Am. Soc. Inf. Sci..

[15]  Karen Markey Drabenstott,et al.  Search trees for subject searching in online catalogs , 1990 .

[16]  Micheline Hancock-Beaulieu,et al.  An Evaluation of Automatic Query Expansion in an Online Library Catalogue , 1992, J. Documentation.

[17]  Michael D. Cooper,et al.  Response time variations in an online search system , 1983, J. Am. Soc. Inf. Sci..

[18]  Shan-Ju L. Chang,et al.  Taking account of User Tasks, Goals and Behavior for the Design of Online Public Access Catalogs , 1990 .

[19]  Raymond G. Taylor,et al.  Utilization of commands in an online bibliographic editing system , 1990 .

[20]  Mark Hinnebusch Performing resource usage analysis for a NOTIS system , 1991 .

[21]  Natalie Schoch King Search Characteristics and the Effects of Experience on End Users of PaperChase , 1991 .

[22]  Brian Nielsen,et al.  What They Say They Do and What They Do: Assessing Online Catalog Use Instruction through Transaction Monitoring. , 1986 .

[23]  C. R. Hildreth,et al.  Intelligent Interfaces and Retrieval Methods for Subject Searching in Bibliographic Retrieval Systems , 1989 .

[24]  Micheline Hancock-Beaulieu,et al.  Evaluating the Impact of an Online Library Catalogue on subject Searching behaviour at the Catalogue and T the shelves , 1991, J. Documentation.

[25]  W. D. Dominick,et al.  A performance measurement and evaluation environment for information systems , 1987, Inf. Process. Manag..

[26]  William H. Mischo,et al.  Interface design and development: the human factor , 1992 .

[27]  Marilyn Ann Lester Coincidence of user vocabulary and Library of Congress subject headings : experiments to improve subject access in academic library online catalogs , 1989 .