Ordinary search engine users carrying out complex search tasks

Web search engines have become the dominant tools for finding information on the Internet. Owing to their popularity, users of all educational backgrounds and professions use them for a wide range of tasks, from simple look-up to rather complex information-seeking needs. This paper presents the results of a study that investigates the behavioural search characteristics of ordinary Web search engines users. The aim of the study was to investigate (1) what makes complex search tasks distinct from simple search tasks and whether it is possible to find simple measures for describing their complexity, and (2) whether successful searchers show different search behaviours than unsuccessful searchers and whether good searchers can be identified via simple measures. The study included 56 ordinary Web users who carried out a set of 12 search tasks using current commercial search engines. Their behaviour was logged with the Search-Logger tool. The results confirm that the behaviour in the case of complex search tasks has significantly different inherent characteristics than in the case of simple search tasks. This can be proven by using simple measures such as task time, number of queries, or number of browser tabs used. We also observed that it is difficult to distinguish successful from unsuccessful search behaviour simply by using these measures. The implications of our findings for search engine vendors are discussed. The results of this study with a sample of ordinary users are insofar unique as they are valid for a wider population while most studies in the field are usually done using convenience samples such as university students.

[1]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[2]  Soo Young Rieh,et al.  Analysis of multiple query reformulations on the web: The interactive information retrieval context , 2006, Information Processing & Management.

[3]  Nadine Höchstötter,et al.  Standard parameters for searching behaviour in search engines and their empirical evaluation , 2009, J. Inf. Sci..

[4]  M. E. Shaw Group dynamics : the psychology of small group behavior , 1971 .

[5]  Dirk Lewandowski,et al.  Query types and search topics of German Web search engine users , 2007, Inf. Serv. Use.

[6]  SpinkAmanda,et al.  How are we searching the world wide web , 2006 .

[7]  Dirk Lewandowski,et al.  Search-logger analyzing exploratory search tasks , 2011, SAC.

[8]  Barry Smyth,et al.  Are people biased in their use of search engines? , 2008, CACM.

[9]  Mike Thelwall,et al.  Synthesis Lectures on Information Concepts, Retrieval, and Services , 2009 .

[10]  A. Maslow Motivation and Personality , 1954 .

[11]  David Robins,et al.  Interactive Information Retrieval: Context and Basic Notions , 2000, Informing Sci. Int. J. an Emerg. Transdiscipl..

[12]  Ben Shneiderman,et al.  Determining Causes and Severity of End-User Frustration , 2004, Int. J. Hum. Comput. Interact..

[13]  Amanda Spink,et al.  Defining a session on Web search engines , 2007, J. Assoc. Inf. Sci. Technol..

[14]  Efthimis N. Efthimiadis,et al.  Analyzing and evaluating query reformulation strategies in web search logs , 2009, CIKM.

[15]  Donald J. Campbell,et al.  The interactive effects of task complexity and participation on task performance: A field experiment , 1986 .

[16]  Pertti Vakkari,et al.  Task-based information searching , 2005, Annu. Rev. Inf. Sci. Technol..

[17]  Christoph Hölscher,et al.  Web search behavior of Internet experts and newbies , 2000, Comput. Networks.

[18]  Marti A. Hearst Search User Interfaces , 2009 .

[19]  Gary Marchionini,et al.  Editorial: Evaluating exploratory search systems , 2008 .

[20]  Andrei Broder,et al.  A taxonomy of web search , 2002, SIGF.

[21]  Dirk Lewandowski,et al.  What Users See - Structures in Search Engine Results Pages , 2009, Inf. Sci..

[22]  Yuan Cheng,et al.  Measuring task complexity in information search from user's perspective , 2011, ASIST.

[23]  Robert G. Capra,et al.  Designing exploratory search tasks for user studies of information seeking support systems , 2009, JCDL '09.

[24]  Amanda Spink,et al.  Patterns of query reformulation during Web searching , 2009, J. Assoc. Inf. Sci. Technol..

[25]  Preben Hansen,et al.  Conceptual framework for tasks in information studies: Book Reviews , 2005 .

[26]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[27]  Judit Bar-Ilan,et al.  Presentation bias is significant in determining user preference for search results—A user study , 2009 .

[28]  Robert G. Capra,et al.  Creating Exploratory Tasks for a Faceted Search Interface , 2008 .

[29]  D. Campbell Task Complexity: A Review and Analysis , 1988 .

[30]  Robert G. Capra HCI Browser: A Tool for Studying Web Search Behavior , 2009, UIIR@SIGIR.

[31]  Alison J. Head,et al.  How college students use the Web to conduct everyday life research , 2011, First Monday.

[32]  Jeffrey Rubin,et al.  Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests , 1994 .

[33]  Ryen W. White,et al.  Exploratory Search: Beyond the Query-Response Paradigm , 2009, Exploratory Search: Beyond the Query-Response Paradigm.

[34]  Peter G. Anick Using terminological feedback for web search refinement: a log-based study , 2003, SIGIR.

[35]  Dietmar Wolfram,et al.  Search characteristics in different types of Web-based IR environments: Are they the same? , 2008, Inf. Process. Manag..

[36]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[37]  Nicholas J. Belkin,et al.  An exploration of the relationships between work task and interactive information search behavior , 2010, J. Assoc. Inf. Sci. Technol..

[38]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[39]  Amanda Spink,et al.  How are we searching the World Wide Web? A comparison of nine search engine transaction logs , 2006, Inf. Process. Manag..

[40]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part II. Results of a Design Study , 1982, J. Documentation.

[41]  Mark Claypool,et al.  Implicit interest indicators , 2001, IUI '01.

[42]  Peter Ingwersen,et al.  Information Retrieval Interaction , 1992 .

[43]  Bernard J. Jansen,et al.  Wrapper: An Application for Evaluating Exploratory Searching Outside of the Lab , 2006 .

[44]  Ryen W. White,et al.  WWW 2007 / Track: Browsers and User Interfaces Session: Personalization Investigating Behavioral Variability in Web Search , 2022 .

[45]  Nicholas J. Belkin,et al.  A faceted approach to conceptualizing tasks in information seeking , 2008, Inf. Process. Manag..

[46]  Preben Hansen,et al.  Conceptual framework for tasks in information studies , 2005, J. Assoc. Inf. Sci. Technol..

[47]  Bettina Berendt,et al.  Report on the TrebleCLEF query log analysis workshop 2009 , 2009, SIGF.

[48]  Nicholas J. Belkin,et al.  Ask for Information Retrieval: Part I. Background and Theory , 1997, J. Documentation.

[49]  Christopher N. Carlson Information overload, retrieval strategies and Internet user empowerment , 2003 .

[50]  Bernard J. Jansen,et al.  Using the taxonomy of cognitive learning to model online searching , 2009, Inf. Process. Manag..

[51]  Michael A. Shepherd,et al.  A Field Study Characterizing Web-based Information Seeking Tasks , 2022 .

[52]  Werner Wirth,et al.  Navigating the Internet , 2004 .

[53]  Ian Ruthven,et al.  Searcher's Assessments of Task Complexity for Web Searching , 2004, ECIR.

[54]  Karen Markey Twenty-five years of end-user searching, Part 1: Research findings , 2007 .

[55]  Linda Langford Surf's up: harnessing information overload , 2010, IEEE Engineering Management Review.

[56]  Ulrich Norbisrath,et al.  Complex search: aggregation, discovery, and synthesis , 2012 .

[57]  Robert W. Reeder,et al.  WebEyeMapper and WebLogger: tools for analyzing eye tracking data collected in web-use studies , 2001, CHI Extended Abstracts.

[58]  Kalervo Järvelin,et al.  Task Complexity Affects Information Seeking and Use , 1995, Inf. Process. Manag..

[59]  Orland Hoeber,et al.  Web Information Retrieval Support Systems: The Future of Web Search , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[60]  Dirk Lewandowski,et al.  The Retrieval Effectiveness of Web Search Engines: Considering Results Descriptions , 2008, J. Documentation.

[61]  Debora Shaw,et al.  Handbook of usability testing: How to plan, design, and conduct effective tests , 1996 .

[62]  Amanda Spink,et al.  Real life information retrieval: a study of user queries on the Web , 1998, SIGF.

[63]  Amanda Spink,et al.  Real life, real users, and real needs: a study and analysis of user queries on the web , 2000, Inf. Process. Manag..