A user-centered design of a web-based interface to bibliographic databases

This thesis reports results of a research study into the usefulness of a user-centred approach for designing information retrieval interfaces. The main objective of the research was to examine the usability of an existing Web-based IR system in order to design a user-centred prototype Web interface. This research used the Web of Science available at http: //wos. mimas. ac. uk. A series of usability experiments was carried out with the Web of Science. The first experiment was carried out using both novice and experienced users to see their performance and satisfaction with the interface. A set of search tasks was obtained from a user survey and was used in the study. The results showed that there were no significant differences in the time taken to complete the tasks, and the number of different search terms used between the two search groups. Novice users were significantly more satisfied with the interface than the experienced group. However, the experienced group was significantly more successful, and made fewer errors than the novice users. The second experiment was conducted on novices' learning and retention with the Web of Science using the same equipment, tasks and environment. The results of the original learning phase of the experiment showed that novices could readily pick up interface functionality when a brief training was provided. However, their retention of search skills weakened over time. Their subjective satisfaction with the interface also diminished from learning to retention. These findings suggested that the fundamental difficulties of searching IR systems still remain with the Web-based version. A heuristic evaluation was carried out to find out the usability problems in the Web of Science interface. Three human factors experts evaluate the interface. The heuristic evaluation was very helpful in identifying some interface design issues for Web IR systems. The most fundamental of these was increasing the match between system and the real world. The results of both the usability testing and the heuristic evaluations served as a baseline for designing a prototype Web interface. The prototype was designed based on a conceptual model of users' information seeking. Various usability evaluation methods were used to test the usability of the prototype system. After each round of testing, the interface was modified in accordance with the test findings. A summative evaluation of the prototype interface showed that both novice and experienced users improved their search performance. Comparative analysis with the earlier usability studies also showed significant improvements in performance and satisfaction with the prototype. These results show that user-centred methods can yield better interface design for IR systems.

[1]  William H. Mischo,et al.  END-USER SEARCHING OF BIBLIOGRAPHIC DATABASES. , 1987 .

[2]  Matthias Hemmje,et al.  LyberWorld—a visualization user interface supporting fulltext retrieval , 1994, SIGIR '94.

[3]  Helen R. Tibbo,et al.  Freestyle vs. Boolean: A Comparison of Partial and Exact Match Retrieval Systems , 1998, Inf. Process. Manag..

[4]  Bipin C. Desai,et al.  CINDI: A Virtual Library Indexing and Discovery System , 1999, Libr. Trends.

[5]  Jonathan Grudin,et al.  The case against user interface consistency , 1989, CACM.

[6]  Alexandra Dimitroff,et al.  Mental Models Theory and Search Outcome in a Bibliographic Retrieval System. , 1992 .

[7]  Wallace Koehler,et al.  FirstSearch and NetFirst--Web and Dial-up Access: Plus Ca Change, Plus C'est la Meme Chose?. , 1996 .

[8]  R. L. Morris,et al.  When does a Mouse become a Rat? or ... Comparing Performance and Preferences in Direct Manipulation and Command Line Environment , 1991, Comput. J..

[9]  B. Shneiderman,et al.  The dynamic HomeFinder: evaluating dynamic queries in a real-estate information exploration system , 1992, SIGIR '92.

[10]  Greg R. Notess Northern light: new search engine for the Web and full-text articles , 1998 .

[11]  Howard R. Turtle Natural language vs. Boolean query evaluation: a comparison of retrieval performance , 1994, SIGIR '94.

[12]  Micheline Hancock-Beaulieu,et al.  Interactive searching and interface issues in the Okapi best match probabilistic retrieval system , 1998, Interact. Comput..

[13]  Deborah J. Mayhew,et al.  Principles and Guidelines in Software User Interface Design , 1991 .

[14]  Deborah Lynne Wiley,et al.  Cited References on the Web: A Review of ISI's Web of Science. , 1998 .

[15]  John C. Thomas,et al.  The humanization of computer interfaces , 1983, CACM.

[16]  Sara J. Czaja,et al.  Age related differences in learning to use a text-editing system , 1989 .

[17]  William Saffady The Status of Library Automation at 2000 , 2000 .

[18]  Jennifer Rowley,et al.  Designing Public Access Systems , 1998 .

[19]  Susan D. Goodall Online help in the real world , 1991, SIGDOC '91.

[20]  Gary Marchionini,et al.  Finding facts vs. browsing knowledge in hypertext systems , 1988, Computer.

[21]  W. David Penniman,et al.  Monitoring and evaluation of on-line information system usage , 1980, Inf. Process. Manag..

[22]  Sidney L. Smith,et al.  The user interface to computer-based information systems: a survey of current software design practice , 1984 .

[23]  Colleen Cool,et al.  Ease of use versus user control: an evaluation of Web and non-Web interfaces of online databases , 2000, Online Inf. Rev..

[24]  Morten Hertzum,et al.  Browsing and querying in online documentation: a study of user interfaces and the interaction process , 1996, TCHI.

[25]  Robert A. Virzi,et al.  Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? , 1992 .

[26]  A. W. Roesler,et al.  What help do users need?: taxonomies for on-line information needs & access methods , 1995, CHI '95.

[27]  Wayne D. Gray,et al.  Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods , 1998, Hum. Comput. Interact..

[28]  Brad A. Myers,et al.  Challenges of HCI design and implementation , 1994, INTR.

[29]  Bev Kennedy Comparing menu systems for end-users: After Dark, Knowledge Index, and FirstSearch , 1993 .

[30]  Daniel B. Horn,et al.  Patterns of entry and correction in large vocabulary continuous speech recognition systems , 1999, CHI '99.

[31]  H. S. Becker,et al.  A novice user's interface to information retrieval systems , 1986, Inf. Process. Manag..

[32]  Cathleen Wharton,et al.  Cognitive Walkthroughs: A Method for Theory-Based Evaluation of User Interfaces , 1992, Int. J. Man Mach. Stud..

[33]  Carol H. Fenichel,et al.  Online searching: Measures that discriminate among users with different types of experiences , 1981, J. Am. Soc. Inf. Sci..

[34]  David V. Canter,et al.  User navigation in complex database systems , 1986 .

[35]  Lorne Olfman,et al.  Conceptual Versus Procedural Software Training for Graphical User Interfaces: A Longitudinal Field Experiment , 1994, MIS Q..

[36]  A. Steven Pollitt,et al.  CANSEARCH: An expert systems approach to document retrieval , 1987, Inf. Process. Manag..

[37]  James E. McDonald,et al.  Searching for Items in Menus: The Effects of Organization and Type of Target , 1983 .

[38]  Christine L. Borgman,et al.  Effective Online Searching: A Basic Text , 1984 .

[39]  Nicholas J. Belkin,et al.  Cases, scripts, and information-seeking strategies: On the design of interactive information retrieval systems , 1995 .

[40]  Peter G. Polson,et al.  An automated cognitive walkthrough , 1991, CHI '91.

[41]  Brad A. Myers,et al.  User interface software tools , 1995, TCHI.

[42]  Vladimir G. Voiskunskii,et al.  Boolean Search: Current State and Perspectives , 1999, J. Am. Soc. Inf. Sci..

[43]  Efthimis N. Efthimiadis,et al.  Interactive query expansion: A user-based evaluation in a relevance feedback environment , 2000, J. Am. Soc. Inf. Sci..

[44]  Richard S. Marcus,et al.  An experimental comparison of the effectiveness of computers and humans as search intermediaries , 1983, J. Am. Soc. Inf. Sci..

[45]  Jane M. Carey,et al.  Pull-down versus traditional menu types: An empirical comparison , 1996, Behav. Inf. Technol..

[46]  Patrick A. Holleran,et al.  A methodological note on pitfalls in usability testing , 1991 .

[47]  D. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[48]  Ben Shneiderman,et al.  Facilitating data exploration with query previews: A study of user performance and preference , 2000, Behav. Inf. Technol..

[49]  C. Borgman From Gutenberg to the global information infrastructure: access to information in the networked world , 2000 .

[50]  William C. Ogden Using Natural Language Interfaces , 1988 .

[51]  L. Whitcomb OCLC's epic system offers a new way to search the OCLC database , 1990 .

[52]  Sidney L. Smith,et al.  Guidelines for Designing User Interface Software , 1986 .

[53]  Jakob Nielsen,et al.  Assessing the usability of a user interface standard , 1991, CHI.

[54]  Nancy A. Van House,et al.  User-Centered Iterative Design for Digital Libraries: The Cypress Experience , 1996, D Lib Mag..

[55]  T. D. Wilson,et al.  Models in information behaviour research , 1999, J. Documentation.

[56]  Bryce Allen,et al.  Topic Knowledge and Online Catalog Search Formulation , 1991, The Library Quarterly.

[57]  Donald T. Hawkins,et al.  Front end software for online database searching part 1: definitions , 1985 .

[58]  Robin Jeffries,et al.  Usability testing vs. heuristic evaluation: was there a contest? , 1992, SGCH.

[59]  Stephen Robertson Online information retrieval systems: an introductory manual to principles and practice ? and ?. Clive Bingley, London. Second Edition (1984). 205 pp. £ 12.50. ISBN: 0-85157-363-0 , 1986 .

[60]  Gary Marchionini,et al.  Bringing Treasures to the Surface: Iterative Design for the Library of Congress National Digital Library Program , 2003 .

[61]  David Orton,et al.  Online searching in science and technology : an introductory guide to equipment, databases and search techniques , 1995 .

[62]  Ben Shneiderman,et al.  Direct Manipulation: A Step Beyond Programming Languages , 1983, Computer.

[63]  Peter Brusilovsky,et al.  Methods and techniques of adaptive hypermedia , 1996, User Modeling and User-Adapted Interaction.

[64]  Clare-Marie Karat,et al.  Comparison of empirical testing and walkthrough methods in user interface evaluation , 1992, CHI.

[65]  Douglas J. Gillan,et al.  Designers' models of the human-computer interface , 1990, CHI '90.

[66]  Ben Shneiderman,et al.  A Study of File Manipulation by Novices Using Commands vs. Direct Manipulation , 1995 .

[67]  Mark V. Springett,et al.  A comparison of usability techniques for evaluating design , 1997, DIS '97.

[68]  Thomas P. Moran,et al.  The Command Language Grammar: A Representation for the User Interface of Interactive Computer Systems , 1981, Int. J. Man Mach. Stud..

[69]  Scott Henninger,et al.  A methodology and tools for applying context-specific usability guidelines to interface design , 2000, Interact. Comput..

[70]  Kevin Mullet,et al.  Designing Visual Interfaces: Communication Oriented Techniques , 1994 .

[71]  Deborah J. Mayhew,et al.  The usability engineering lifecycle , 1999, CHI Extended Abstracts.

[72]  Philip J. Smith Understanding interfaces: A handbook of human-computer interaction: M. W. Lonsdale and T. C. Ormerod. Academic Press, London (1994). xi + 289 pp., $45 hardback. ISBN 0-12-528390-3 , 1995 .

[73]  Jakob Nielsen,et al.  Finding usability problems through heuristic evaluation , 1992, CHI.

[74]  Robert M. Losee,et al.  Measuring search-engine quality and query difficulty: ranking with Target and Freestyle , 1999 .

[75]  Charles T. Meadow,et al.  The design and evaluation of a front-end user interface for energy researchers , 1989, JASIS.

[76]  W. Bruce Croft,et al.  I3R: A new approach to the design of document retrieval systems , 1987, J. Am. Soc. Inf. Sci..

[77]  Ben Shneiderman,et al.  Speech versus Mouse Commands for Word Processing: An Empirical Evaluation , 1993, Int. J. Man Mach. Stud..

[78]  David R. Schwartz,et al.  The use of guidelines in interface design , 1991, CHI.

[79]  Kathleen K. Molnar,et al.  The impacts on user performance and satisfaction of a voice-based front-end interface for a standard software tool , 1996, Int. J. Hum. Comput. Stud..

[80]  Cathleen Wharton,et al.  The cognitive walkthrough method: a practitioner's guide , 1994 .

[81]  Matthew Chalmers,et al.  Bead: explorations in information visualization , 1992, SIGIR '92.

[82]  F. W. Lancaster Toward paperless information systems , 1978 .

[83]  Randy Pausch,et al.  A Study Comparing Mouse-Only Input for a Graphical Editor , 1990 .

[84]  Martha E. Williams,et al.  Online Retrieval--Today and Tomorrow. , 1978 .

[85]  Helen M. Brooks,et al.  Plexus-the expert system for referral , 1987, Inf. Process. Manag..

[86]  Ben Shneiderman,et al.  Dynamic queries for visual information seeking , 1994, IEEE Software.

[87]  Kyle L. Peck,et al.  Pull-Down Menus, Menu Design, and Usage Patterns in Computer-Assisted Instruction. , 1991 .

[88]  Jennifer Jerrams-Smith,et al.  An Attempt to Incorporate Expertise about Users into an Intelligent Interface for Unix , 1989, Int. J. Man Mach. Stud..

[89]  Ben Shneiderman,et al.  Clarifying Search: A User-Interface Framework for Text Searches , 1997, D Lib Mag..

[90]  J. L. Alty,et al.  How useful is on-line help?: An observational study , 1998, Proceedings 1998 Australasian Computer Human Interaction Conference. OzCHI'98 (Cat. No.98EX234).

[91]  Micheline Hancock-Beaulieu,et al.  An Evaluation of Interactive Query Expansion in an Online Library Catalogue with a Graphical User Interface , 1995, J. Documentation.

[92]  Jakob Nielsen,et al.  Iterative user-interface design , 1993, Computer.

[93]  Liwen Qiu,et al.  Markov Models of Search State Patterns in a Hypertext Information Retrieval System , 1993, J. Am. Soc. Inf. Sci..

[94]  Keith Oatley,et al.  Measuring the learnability of spreadsheets in inexperienced users and those with previous spreadsheet experience , 1991 .

[95]  Donna K. Harman,et al.  Relevance feedback revisited , 1992, SIGIR '92.

[96]  Cathleen Wharton,et al.  Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces , 1990, CHI '90.

[97]  Bryce Allen,et al.  Information Tasks: Toward a User-Centered Approach to Information Systems , 1996 .

[98]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[99]  Stephen P. Harter,et al.  Online Information Retrieval: Concepts, Principles and Techniques , 1986 .

[100]  Christopher Williamson,et al.  Dynamic queries for information exploration: an implementation and evaluation , 1992, CHI.

[101]  Gregg Skip Bailey,et al.  Iterative methodology and designer training in human-computer interface design , 1993, INTERCHI.

[102]  Amanda Spink,et al.  Real life, real users, and real needs: a study and analysis of user queries on the web , 2000, Inf. Process. Manag..

[103]  Weijing Yuan,et al.  End-User Searching Behavior in Information Retrieval: A Longitudinal Study , 1997, J. Am. Soc. Inf. Sci..

[104]  John I. Kiger,et al.  The Depth/Breadth Trade-Off in the Design of Menu-Driven User Interfaces , 1984, Int. J. Man Mach. Stud..

[105]  Gary Marchionini,et al.  Design of Interfaces for Information Seeking. , 1998 .

[106]  Izak Benbasat,et al.  An Experimental Investigation of Interface Design Alternatives: Icon vs. Text and Direct Manipulation vs. Menus , 1993, Int. J. Man Mach. Stud..

[107]  John M. Carroll,et al.  Learning by doing with simulated intelligent help , 1988, CACM.

[108]  Andrew Dillon,et al.  User analysis in HCI - the historical lessons from individual differences research , 1996, Int. J. Hum. Comput. Stud..

[109]  Ben Shneiderman,et al.  Sorting out searching: a user-interface framework for text searches , 1998, CACM.

[110]  Ben Shneiderman,et al.  Visual and Textual Consistency Checking Tools for Graphical User Interfaces , 1997, IEEE Trans. Software Eng..

[111]  Ramana Rao,et al.  The table lens: merging graphical and symbolic representations in an interactive focus + context visualization for tabular information , 1994, CHI '94.

[112]  Donald T. Hawkins,et al.  End-user searching in a large library network: a case study of patent attorneys , 1986 .

[113]  Amanda Spink Online retrieval: A dialogue of theory and practice , 1994 .

[114]  Alistair G. Sutcliffe,et al.  Empirical studies of end-user information searching , 2000, J. Am. Soc. Inf. Sci..

[115]  Alistair G. Sutcliffe,et al.  Towards a cognitive theory of information retrieval , 1998, Interact. Comput..

[116]  R. G. Crawford,et al.  A Prototype Mouse-Based Interface to Drug Related Information. , 1985 .

[117]  John Cox,et al.  Keyguide to Information Sources in Online and Cd-Rom Database Searching , 1991 .

[118]  Steve Jones Graphical query specification and dynamic result previews for a digital library , 1998, UIST '98.

[119]  David M. Lane,et al.  Impact of a restricted natural language interface on ease of learning and productivity , 1989, CACM.

[120]  D. Cramer,et al.  Introduction to Statistics in Psychology , 2005 .

[121]  Donna L. Cuomo,et al.  Understanding Usability Issues Addressed by Three User-System Interface Evaluation Techniques , 1994, Interact. Comput..

[122]  John M. Carroll,et al.  Interface metaphors and user interface design , 1988 .

[123]  Gary Perlman,et al.  The FirstSearch user interface architecture: universal access for any user, in many languages, on any platform , 2000, CUU '00.

[124]  Greg Kearsley Online help systems: design and implementation , 1988 .

[125]  S. O. Parsons,et al.  Human Factors Society , 1966 .

[126]  Micheline Hancock-Beaulieu,et al.  An Evaluation of Automatic Query Expansion in an Online Library Catalogue , 1992, J. Documentation.

[127]  K. Hirsh-Pasek,et al.  An experimental evaluation of abbreviation schemes in limited lexicons , 1982 .

[128]  Susan M. Harrison A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface , 1995, CHI '95.

[129]  Robert P. Bostrom,et al.  Training End Users: An Experimental Investigation of the Roles of the Computer Interface and Training Methods , 1993, MIS Q..

[130]  K. Potosnak Recipe for a Usability Test , 1988, IEEE Softw..

[131]  A. L. Barker DataStar Web : living up to the hype? An evaluation of the interface and search system , 1997 .

[132]  Judith S. Olson,et al.  A mental model can help with learning to operate a complex device , 1993, CHI '93.

[133]  Ben Shneiderman,et al.  Interface and data architecture for query preview in networked information systems , 1999, TOIS.

[134]  Donald A. Norman,et al.  Some observations on mental models , 1987 .

[135]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[136]  Nicholas J. Belkin,et al.  Interface issues and interaction strategies for information retrieval systems , 1995, CHI '95.

[137]  Brad A. Myers,et al.  Past, Present and Future of User Interface Software Tools , 2000, TCHI.

[138]  Jurek Kirakowski,et al.  Measuring the Usability of Web Sites , 1998 .

[139]  Ellen D. Hoadley Investigating the effects of color , 1990, CACM.

[140]  Gavriel Salvendy,et al.  Hierarchical Menu Design: Breadth, Depth, and Task Complexity , 1996 .

[141]  Matthias Jarke,et al.  A Field Evaluation of Natural Language for Data Retrieval , 1983, IEEE Transactions on Software Engineering.

[142]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[143]  Anne L. Barker Datastar Web: a Comparison with "Classic" DataStar Command Language Searching , 1998, Online Inf. Rev..

[144]  Robert C. Williges,et al.  Retrieval of HELP Information for Novice Users of Interactive Computer Systems , 1985 .