Usage and Usability Assessment: Library Practices and Concerns

This 2002 report offers a survey of the methods deployed at leading digital libraries to assess the use and usability of their online collections and services. Focusing on 24 Digital Library Federation member libraries, author Denise Troll Covey conducted numerous interviews with library professionals engaged in assessment.

[1]  Linda A. Brown,et al.  The evolving approval plan: how academic librarians evaluate services for vendor selection and performance , 1999 .

[2]  Patricia Te Arapo Wallace,et al.  How Do Patrons Search the Online Catalog When No One's Looking? Transaction Log Analysis and Implications for Bibliographic Instruction and Systems Design. , 1993 .

[3]  Tom Peters,et al.  The history and development of transaction log analysis , 1993 .

[4]  Richard L. Hart,et al.  Co-authorship in the academic library literature: A survey of attitudes and behaviors , 2000 .

[5]  D. Dillman.  Mail and telephone surveys : the total design method , 1979 .

[6]  King Ns.  End-user errors: a content analysis of PaperChase transaction logs , 1993 .

[7]  Joseph Janes,et al.  Digital Reference Services in Academic Libraries , 1999 .

[8]  Leslie Horner Button,et al.  Impact of Bundled Databases on Serials Acquisitions in Academic Libraries , 2000 .

[9]  Anne C. Ciliberti,et al.  Empty handed? A material availability study and transaction log analysis verification , 1998 .

[10]  Gayle Baker,et al.  A Heuristic Approach to Selecting Delivery Mechanisms for Electronic Resources in Academic Libraries. , 1999 .

[11]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[12]  Cheryl R.S Grossman,et al.  Electronic Journals in Academic Libraries: A Comparison of ARL and Non-ARL Libraries. , 1999 .

[13]  Amy Smith,et al.  Factors for Success: Academic Library Development Survey Results , 2000, Libr. Trends.

[14]  Richard A. Krueger.  Focus Groups: A Practical Guide for Applied Research, Second Edition , 1994 .

[15]  Alison J. Head,et al.  Web redemption and the promise of usability , 1999 .

[16]  Brenda L. Battleson,et al.  Usability testing of an academic library Web site: a case study , 2001 .

[17]  Andrew Dillon,et al.  Artifacts as Theories: Convergence through User-Centered Design. , 1995 .

[18]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[19]  Janet Mancini Billson,et al.  Focus Groups: A Practical Guide for Applied Research , 1989 .

[20]  Beryl Glitz,et al.  Focus Groups for Libraries and Librarians , 1998 .

[21]  Jakob Nielsen,et al.  Designing web usability , 1999 .

[22]  Richard A. Krueger,et al.  Developing Questions for Focus Groups , 1997 .

[23]  Jaquelina E. Álvarez,et al.  Internet research : The role of the focus group , 2000 .

[24]  Marisa Scigliano.  Serial Use in a Small Academic Library: Determining Cost-Effectiveness , 2000 .

[25]  Soyeon Park,et al.  Usability, user preferences, effectiveness, and user behaviors when searching individual and integrated full-text databases: implications for digital libraries , 2000, J. Am. Soc. Inf. Sci..

[26]  Brendan J. Wyly.  From Access Points to Materials: A Transaction Log Analysis of Access Point Value for Online Catalog Users , 1996 .

[27]  Darlene Fichter.  Testing the web site usuability waters , 2001 .

[28]  O. Gene Norman.  The impact of electronic information sources on collection development: a survey of current practice , 1997 .

[29]  Jakob Nielsen,et al.  Heuristic Evaluation of Prototypes (individual) , 2022 .

[30]  Lois Buttlar,et al.  Catalogers in Academic Libraries: Their Evolving and Expanding Roles. , 1998 .

[31]  Mary Beth Allen,et al.  International Students in Academic Libraries: A User Survey , 1993 .

[32]  Fay Sudweeks,et al.  Culture, technology, communication : towards an intercultural global village , 2001 .

[33]  Myke Gluck.  The application of the usability approach in libraries and information centers for resource selection and deployment , 1998 .

[34]  Theresa Dirndorfer Anderson,et al.  Searching for Information: Applying Usability Testing Methods to a Study of Information Retrieval and Relevance Assessment , 1999 .

[36]  Laura Hudson.  Radical usability (or, why you need to stop redesigning your Web site) , 2000 .

[37]  Elaine Toms,et al.  The impact of information architecture on academic web site usability , 1999, Electron. Libr..

[38]  Barbara P. Buttenfield,et al.  Usability Evaluation of Digital Libraries , 1999 .

[39]  Terry Ellen Ferl,et al.  The Knuckle-Cracker's Dilemma: A Transaction Log Study of OPAC Subject Searching. , 1996 .

[40]  M. White.  Diffusion of an innovation: digital reference service in Carnegie Foundation master’s (comprehensive) academic institution libraries , 2001 .

[41]  Tina Schneider.  The regional campus library and service to the public , 2001 .

[42]  Charles L. Weber,et al.  A Mathematical Model , 1987 .

[43]  Heidi E. Julien.  Information Literacy Instruction in Canadian Academic Libraries: Longitudinal Trends and International Comparisons , 2000 .

[44]  Thomas Pack.  Use it or lose it: Jakob Nielsen champions content usability , 2001 .

[45]  Michel C. Atlas,et al.  Flip charts at the OPAC: using transaction log analysis to judge their effectiveness , 1997 .

[46]  Bryce Allen,et al.  Cognitive Abilities and Information System Usability , 1994, Inf. Process. Manag..

[47]  Jakob Nielsen,et al.  A mathematical model of the finding of usability problems , 1993, INTERCHI.

[48]  Frederick G. Conrad,et al.  Usability testing of world wide web sites , 1997, CHI Extended Abstracts.

[49]  The perceptions of library service questionnaire (PLSQ): The development of a reliable instrument to measure student perceptions of and satisfaction with quality of service in an academic library , 1995 .

[50]  H. Russell Bernard,et al.  Social Research Methods: Qualitative and Quantitative Approaches , 2000 .

[51]  Philip Calvert,et al.  Surveying Service Quality within University Libraries. , 1997 .

[52]  Julie L Rabine,et al.  How Libraries Are Providing Access to Electronic Serials: A Survey of Academic Library Web Sites , 1999 .

[53]  Steve Walker,et al.  Transaction logging , 1997, J. Documentation.

[54]  Robert O. Keohane,et al.  Designing Social Inquiry: Scientific Inference in Qualitative Research. , 1995 .

[55]  Michael J. Prasse,et al.  Chauffeured by the User: Usability in the Electronic Library at the University of Arizona and OCLC , 1999 .

[56]  Michael Coffta,et al.  Academic Library Web Sites as a Source of Interlibrary Loan Lending Information , 2000 .

[57]  E. Morgan.  Marketing Through Usability , 1999 .

[58]  G. Morgan,et al.  Beyond Method: Strategies for Social Research. , 1984 .

[59]  Larry Millsap,et al.  Search Patterns of Remote Users: An Analysis of OPAC Transaction Logs. , 1993 .

[60]  Ben Shneiderman,et al.  Perspective-based Usability Inspection: An Empirical Validation of Efficacy , 1999, Empirical Software Engineering.

[61]  J. Pavy.  Special Collections in Regional Academic Libraries: An Informal Survey , 2000 .

[62]  Elaine G. Toms,et al.  Usability of the Academic Library Web Site: Implications for Design , 2001 .

[63]  Janet M. Schneider.  Focus Groups for Libraries and Librarians. , 2000 .

[64]  Ruth A. Palmquist.  An Overview of Usability for the Study of Users' Web- Based Information Retrieval Behavior , 2001 .

[65]  Jakob Nielsen,et al.  Paper versus computer implementations as mockup scenarios for heuristic evaluation , 1990, INTERACT.

[66]  Lynn Silipigni Connaway.  Focus Group Interviews: A Data Collection Methodology. , 1996 .

[67]  Patricia Davitt Maughan,et al.  Library resources and services: a cross-disciplinary survey of faculty and graduate student use and satisfaction , 1999 .

[68]  N S King.  End-user errors: a content analysis of PaperChase transaction logs. , 1993, Bulletin of the Medical Library Association.

[69]  Neal K. Kaske.  Research methodologies and transaction log analysis: Issues, questions, and a proposed model , 1993 .

[70]  Michael B. Spring,et al.  Online image databases: usability and performance , 1995 .

[71]  Beth Sandore,et al.  A manifesto ‐ regarding the future of transaction log analysis , 1993 .

[72]  Jakob Nielsen,et al.  Enhancing the explanatory power of usability heuristics , 1994, CHI '94.

[73]  L. Ellis.  Research Methods in the Social Sciences , 1994 .

[74]  Wendy A. Rogers,et al.  Assessing the usability of on-line library systems , 1998, Behav. Inf. Technol..

[75]  Jakob Nielsen,et al.  Improving a human-computer dialogue , 1990, CACM.

[76]  Robin N. Sinn.  A comparison of library instruction content by biology faculty and librarians , 2000 .

[77]  Beth Sandore,et al.  Applying the results of transaction log analysis , 1993 .

[78]  Robin Jeffries,et al.  User interface evaluation in the real world: a comparison of four techniques , 1991, CHI.

[79]  Susan Davis Herring.  Using the World Wide Web for research: are faculty satisfied? , 2001 .

[80]  Richard A. Krueger,et al.  Involving Community Members in Focus Groups , 1997 .

[82]  B. Chadwick,et al.  Social Science Research Methods , 1984 .

[83]  David L. Morgan,et al.  Planning Focus Groups , 1997 .

[84]  Joanna Bawa,et al.  The politics of usability - a practical guide to designing usable systems in industry , 1998, BCS practitioner series.

[85]  Sever Bordeianu,et al.  Cataloging Production Standards in Academic Libraries , 1999 .

[86]  Audrey F. Bancroft,et al.  A forward-looking library use survey: WSU libraries in the 21st century , 1998 .

[87]  Jennifer L Branch.  Investigating the Information-Seeking Processes of Adolescents: The Value of Using Think Alouds and Think Afters , 2000 .

[88]  Margie H. Jantti,et al.  Academic Library Seating: A Survey of Usage, with Implications for Space Utilisation , 1997 .

[89]  Ruth Dickstein,et al.  Usability testing at the University of Arizona library : How to let the users in on the design , 2000 .

[90]  Janet Chisman,et al.  Discovering the User: A Practical Glance at Usability Testing. , 1999 .

[91]  K. Weick.  Mail and telephone surveys: The total design method. , 1978 .

[92]  Heather Morrison,et al.  Information Literacy Skills: an exploratory focus group study of student perceptions , 1997 .

[93]  Beth Sandore,et al.  Transaction log analysis , 1993 .

[94]  J. M. Budd,et al.  An investigation of the use of an online catalog : user characteristics and transaction log analysis , 1995 .

[95]  D. Morgan.  Focus groups for qualitative research. , 1988, Hospital guest relations report.

[96]  Richard Rubin,et al.  Focus groups: A practical guide for applied research, second edition , 1996 .

[97]  Philip Calvert,et al.  Methods for measuring service quality in University Libraries in New Zealand , 1996 .

[98]  Jane Farley Templeton,et al.  Focus group : a strategic guide to organizing, conducting and analyzing the focus group interview , 1994 .

[99]  X. Bao.  Academic library home pages: link location and database provision , 2000 .

[100]  H. Edmunds.  The Focus Group Research Handbook , 1999 .

[101]  D. Morgan.  Successful Focus Groups: Advancing the State of the Art , 1993 .

[102]  Wen-Hua Ren,et al.  Library instruction and college student self-efficacy in electronic information searching , 2000 .

[103]  Bob Travica,et al.  Organizational aspects of the virtual library: A survey of academic libraries , 1999 .

[104]  Frank R. Allen,et al.  Materials Budgets in the Electronic Age: A Survey of Academic Libraries , 1996 .

[105]  David Benyon,et al.  Integrating usability into systems development. , 1998 .

[106]  Susan E. Searing,et al.  Online Catalogs from the Users' Perspective: The Use of Focus Group Interviews. , 1997 .

[107]  Janet Chisman,et al.  Usability Testing: A Case Study , 1999 .

[108]  Beth Sandore,et al.  An introduction to the special section on transaction log analysis , 1993 .

[109]  Michael J. Wenger,et al.  Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests by Jeffrey Rubin 330 pages, $34.95 New York: John Wiley & Sons, 1994 ISBN 0-471-59403-2 , 1995 .

[110]  N. Dubin.  Mathematical Model , 2022 .

[111]  Mark Hawkes.  Methods of educational and social science research: An integrated approach , 1994 .

[112]  Carol Hansen Montgomery,et al.  Framework for assessing the impact of an electronic journal collection on library costs and staffing patterns , 2000 .

[113]  Diana D. Shonrock.  Evaluating Library Instruction: Sample Questions, Forms, and Strategies for Practical Use , 1996 .

[114]  John Feather,et al.  Special collections on the World Wide Web a survey and evaluation , 1998 .

[115]  B. Glitz.  The focus group technique in library research: an introduction. , 1997, Bulletin of the Medical Library Association.

[116]  Norman M. Bradburn,et al.  DO RESPONDENTS REALLY “MARK ALL THAT APPLY” ON SELF-ADMINISTERED QUESTIONS? , 1994 .

[117]  Richard A. Krueger,et al.  Moderating Focus Groups , 1997 .

[118]  Susan Wiedenbeck,et al.  Using protocol analysis to study the user interface , 1989 .

[119]  Gayle Porter,et al.  Nonprint Formats: A Survey of the Work and Its Challenges for the Cataloger in ARL Academic Libraries , 1997 .

[120]  Heather Morrison.  Online catalogue research and the verbal protocol method , 1999 .

[121]  M. Budden.  Focus groups: Theory and practice , 1999 .

[122]  Ann C. Weller,et al.  Using Transaction Log Analysis to Improve OPAC Retrieval Results , 1998 .

[123]  Carol A. Hert,et al.  A usability assessment of online indexing structures in the networked environment , 2000, J. Am. Soc. Inf. Sci..

[124]  Debora Shaw,et al.  Handbook of usability testing: How to plan, design, and conduct effective tests , 1996 .

[125]  Paula Sullenger.  A serials transaction log analysis , 1997 .

[126]  Giacomo Mauro DAriano.  The Handbook for Focus Group Research. , 1994 .

[127]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[128]  Maria Elena Sanchez,et al.  EFFECTS OF QUESTIONNAIRE DESIGN ON THE QUALITY OF SURVEY DATA , 1992 .

[129]  D. Morgan.  The Focus Group Guidebook , 1997 .

[130]  Bob Travica,et al.  Organizational Aspects of the Virtual/Digital Library: A Survey of Academic Libraries. , 1997 .

[131]  Toby Chance.  Ensuring online information usability , 1993 .

[132]  Martin Kurth,et al.  The limits and limitations of transaction log analysis , 1993 .

[133]  Kerry Martin,et al.  From the users' perspective—The UCSD libraries user survey project , 1998 .

[134]  Richard A. Krueger,et al.  Analyzing and Reporting Focus Group Results , 1997 .

[135]  Susan C. Olason.  Let's get usable! Usability studies for indexes , 2000 .