Using the repertory grid and laddering technique to determine the user's evaluative model of search engines

Purpose – The purpose of this research is to explore a method for the determination of users' representations of search engines, formed during their interaction with these systems. Determines the extent to which these elicited “mental models” indicate the system aspects of importance to the user and from this their evaluative view of these tools.Design/methodology/approach – The repertory grid technique is used to elicit a set of constructs that define facets within the mental model of an individual. A related technique of laddering then considers each of the user's constructs to determine the reasons for its importance within the user's mental model.Findings – The model derived from the qualitative data comprises three hierarchical strata and conveys the interrelations between basic system description, evaluative description, and the key evaluations of ease, efficiency, effort and effectiveness. Two additional layers relating to the perceived process and the experience of emotion are also discussed.Resea...

[1]  Amanda Spink,et al.  Use of query reformulation and relevance feedback by Excite users , 2000, Internet Res..

[2]  G. Kelly.  The Psychology of Personal Constructs , 2020 .

[3]  Amanda Spink,et al.  Searching the Web: a survey of EXCITE users , 1999, Internet Res..

[4]  Amanda Spink,et al.  Searching the Web: the public and their queries , 2001 .

[5]  Debra J. Slone.  The influence of mental models and goals on search patterns during Web interaction , 2002, J. Assoc. Inf. Sci. Technol..

[6]  Andrew Thatcher,et al.  Mental models of the Internet , 1998 .

[7]  M. Gordon Hunter,et al.  The use of RepGrids to gather interview data about information systems analysts , 1997, Inf. Syst. J..

[8]  Frances C. Johnson,et al.  Devise: a framework for the evaluation of internet search engines , 2001 .

[9]  Nigel Shadbolt,et al.  Laddering: technique and tool use in knowledge acquisition , 1994 .

[10]  Amanda Spink,et al.  A user-centered approach to evaluating human interaction with Web search engines: an exploratory study , 2002, Inf. Process. Manag..

[11]  Donald A. Norman,et al.  Some observations on mental models , 1987 .

[12]  Mary Burke.  The Use of Repertory Grids To Develop a User-Driven Classification of a Collection of Digitized Photographs. , 2001 .

[13]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[14]  Frances C. Johnson,et al.  Using the information seeker to elicit construct models for search engine evaluation , 2004, J. Assoc. Inf. Sci. Technol..

[15]  Mark Chignell,et al.  Assessment of the effects of user characteristics on mental models of information retrieval systems , 2001 .

[16]  Monika Henzinger,et al.  Analysis of a very large web search engine query log , 1999, SIGF.

[17]  Christoph Hölscher,et al.  Web search behavior of Internet experts and newbies , 2000, Comput. Networks.

[18]  Felix B. Tan,et al.  The Repertory Grid Technique: A Method for the Study of Cognition in Information Systems , 2002, MIS Q..

[19]  Tony Moynihan,et al.  An inventory of personal constructs for information systems project risk researchers , 1996, J. Inf. Technol..

[20]  Andrew Large,et al.  Users' perceptions of the Web as revealed by transaction log analysis , 2001, Online Inf. Rev..

[21]  Mark D. Dunlop,et al.  New IR - New Evaluation: the impact of interaction and multimedia on information retrieval and its evaluation , 1997, New Rev. Hypermedia Multim..

[22]  Richard Bell,et al.  A manual for repertory grid technique , 1977 .

[23]  Andrew Dillon,et al.  Towards a Classification of Text Types: A Repertory Grid Approach , 1990, Int. J. Man Mach. Stud..

[24]  Andrew Dillon,et al.  Designing Usable Electronic Text: Ergonomic Aspects Of Human Information Usage , 1994 .

[25]  Frances C. Johnson,et al.  Task dimensions of user evaluations of information retrieval systems , 2003, Inf. Res..

[26]  Steve Townend,et al.  Review of: Muhr, T. 1997. ATLAS.ti 5: The Knowledge Workbench. Berlin: Scientific Software Development , 2003 .

[27]  Hongyan Ma.  User satisfaction versus retrieval effectiveness in interactive retrieval , 2008, ASIST.

[28]  Michael Seadle.  Mental models for personal digital assistants (PDAs) , 2003 .

[29]  Nancy Staggers,et al.  Mental Models: Concepts for Human-Computer Interaction Research , 1993, Int. J. Man Mach. Stud..

[30]  Malcolm Otter,et al.  Lost in hyperspace: metrics and mental models , 2000, Interact. Comput..

[31]  Hong Xie,et al.  Supporting ease-of-use and user control: desired features and structure of Web-based online IR systems , 2003, Inf. Process. Manag..

[32]  Gerrit C. van der Veer.  Individual differences and the user interface , 1989 .

[33]  Wanda Pratt,et al.  Transparent Queries: investigation users' mental models of search engines , 2001, SIGIR '01.

[34]  Marc Hassenzahl,et al.  Analysis of web sites with the repertory grid technique , 2001, CHI Extended Abstracts.

[35]  T. J. Reynolds,et al.  Laddering theory, method, analysis, and interpretation. , 2001 .

[36]  A. Strauss,et al.  Basics of Qualitative Research , 1992 .

[37]  Louise T. Su.  A comprehensive and systematic model of user evaluation of Web search engines: I. Theory and background , 2003, J. Assoc. Inf. Sci. Technol..

[38]  A. Bytheway,et al.  Factors affecting information systems’ success , 1996 .

[39]  Lorna Uden,et al.  Insight into mental models of novice Internet searchers , 2003, CACM.

[40]  A. Landfield.  Personal construct systems in psychotherapy , 1971 .

[41]  D. N. Hinkle.  The change of personal constructs from the viewpoint of a theory of construct implications , 1965 .

[42]  James H. Fetzer.  The argument for mental models is unsound , 1993 .

[43]  Lee Ratzan.  Making sense of the web: a metaphorical approach , 2000, Inf. Res..

[44]  Amanda Spink,et al.  Real life, real users, and real needs: a study and analysis of user queries on the web , 2000, Inf. Process. Manag..

[45]  A. Stewart,et al.  Business applications of repertory grid , 1981 .

[46]  Cliff McKnight.  The personal construction of information space , 2000 .