Modeling Result-List Searching in the World Wide Web: The Role of Relevance Topologies and Trust Bias

Modeling Result-List Searching in the World Wide Web: The Role of Relevance Topologies and Trust Bias Maeve O’Brien (maeve.m.obrien@ucd.ie) Mark T. Keane (mark.keane@ucd.ie) Adaptive Information Cluster, University College Dublin, School of Computer Science and Informatics, University College Dublin, Ireland. Abstract There are important cognitive issues surrounding the searching of lists of results returned to search engine queries that could significantly impact system and interface design. In this paper, we focus on result-list search examining two key issues: the influence of relevance topology of the list on first- click behavior, and the question of whether trust-bias occurs in such search. On both issues we advance some empirical and modeling results. These results are discussed in terms of their practical implications for Web designers and practitioners generally. Keywords: search behavior; information predictive user modeling; empirical tests. navigation; Introduction The World Wide Web (WWW; Berners-Lee, T. Cailliau R. Groff J. & Pollermann B.,1992) has presented people with a whole new medium in which to search for information and, arguably, has transformed list-searching from a rather arcane, laboratory phenomenon into a ubiquitous cognitive act. Currently, there are two predominant modes for locating information on the World Wide Web (WWW): browsing and searching. Browsing is the process of viewing pages one at a time and navigating between them sequentially using hyperlinks. Searching refers to the entering of a search query (usually a list of one or more keywords) to a search engine and the subsequent scanning and selection of links from the results returned. Researchers have developed a number of models of Internet browsing (see Brumby & Howes, 2004; Cox & Young, 2004; Miller & Remington, 2005; Pirolli, 2005; Pirolli & Card, 1999; Pirolli & Fu, 2003). Result-list search has received less attention in the Web context, though for decades it has been a mainstay of memory and attention experiments (see Eysenck & Keane, 2000). There are significant issues surrounding how people search result lists within the Web context. In this paper, we focus on two key issues: the significance of the relevance topology of the list, and the issue of trust bias in search engines. On both issues we advance new empirical and modeling results. It has repeatedly been shown that people tend to favor items presented at the top of lists, an effect that has been replicated in the Web context (Joachims, Granka, Pang, Hembrooke, Gay, 2005; Keane, O’Brien & Smyth, in press). Keane et al. (in press), for example, showed, using a simulated Google (Brin & Page, 1998) interface, that when result-lists are systematically reversed in response to user queries, people tend to choose less-relevant results at the beginning of the list over highly-relevant results lower down the list. This bias raises concerns surrounding the search-engines power to route traffic: highly-ranked pages typically benefit from a greater volume of traffic, and this heightened exposure obviously increases the volume of incoming links these top pages receive over time, which in turn increases their ranking prominence and the volume of traffic they receive etc. resulting in a rich-get-richer scenario (Baeza- Yates, Saint-Jean, Castillo, 2002; Cho & Adams, 2003; Cho, Roy, 2004). A key issue surrounding this bias effect is whether it is specifically due to some level of trust in the particular search engine. For example, as people come to trust the fact that Google tend to deliver relevant links in the first three results, people may stop closely assessing results and just lazily click on these first links (c.f., Joachims et al., 2005). An obvious way to check this is to see whether the effects found by Keane et al. (in press) for the simulated Google interface, are also found when the same materials are presented as simple text lists. In their study Keane et al (in press) found that whilst people generally tended to click on top results, there were instances where highly-relevant results lower down the list were clicked. This effect may be due to the different relevance distributions, or topologies, of result-lists. That is, a highly-relevant result proceeding many irrelevant results may stand a greater chance of being chosen over the same highly-relevant result proceeded by other relatively relevant results. In menu searching, Brumby & Howes (2004) have already shown that the extent to which people search through a list interacts with such relevance topologies. But, it is not clear whether such effects extend to search-engine result lists. In this paper, we present an experiment and a model that investigate these two issues. In the experiment we systematically manipulated the relevance topologies of the presented lists, presenting them in either a Google interface or a text-list interface.