A Task-Based Evaluation of an Aggregated Search Interface

This paper presents a user study that evaluated the effectiveness of an aggregated search interface in the context of non-navigational search tasks. An experimental system was developed to present search results aggregated from multiple information sources, and compared to a conventional tabbed interface. Sixteen participants were recruited to evaluate the performance of the two interfaces. Our results suggest that the aggregated search interface is a promising way of supporting non-navigational search tasks. The quantity and diversity of the retrieved items which participants accessed to complete a task, increased in the aggregated interface. Participants also found the aggregated presentation easier to access to retrieved items and to find relevant information, compared to the conventional interface.

[1]  Fernando Diaz,et al.  Integration of news content into web results , 2009, WSDM '09.

[2]  Peter Ingwersen,et al.  Developing a Test Collection for the Evaluation of Integrated Search , 2010, ECIR.

[3]  Jiayu Tang,et al.  What Else Is There? Search Diversity Examined , 2009, ECIR.

[4]  Milad Shokouhi,et al.  Updating collection representations for federated search , 2007, SIGIR.

[5]  Xue Dong Yang,et al.  User-Oriented Evaluation Methods for Interactive Web Search Interfaces , 2007, 2007 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology - Workshops.

[6]  Kevin Li,et al.  Faceted metadata for image search and browsing , 2003, CHI '03.

[7]  Amanda Spink,et al.  Determining the informational, navigational, and transactional intent of Web queries , 2008, Inf. Process. Manag..

[8]  Oren Etzioni,et al.  Grouper: A Dynamic Clustering Interface to Web Search Results , 1999, Comput. Networks.

[9]  Jamie Callan,et al.  DISTRIBUTED INFORMATION RETRIEVAL , 2002 .

[10]  Stephen E. Robertson,et al.  Deciphering cluster representations , 2001, Inf. Process. Manag..

[11]  Filip Radlinski,et al.  Improving personalized web search using result diversification , 2006, SIGIR.

[12]  Pia Borlund,et al.  Experimental components for the evaluation of interactive information retrieval systems , 2000, J. Documentation.

[13]  Joemon M. Jose,et al.  Understanding domain "relevance" in web search , 2009 .

[14]  Tomi Heimonen,et al.  Comparing the User Experience of Search User Interface Designs , 2008 .

[15]  Susan T. Dumais,et al.  Optimizing search by showing results in context , 2001, CHI.

[16]  Ryen W. White,et al.  Evaluating advanced search interfaces using established information-seeking models , 2009, J. Assoc. Inf. Sci. Technol..

[17]  Ryen W. White,et al.  Evaluating advanced search interfaces using established information-seeking models , 2009 .

[18]  Micheline Beaulieu,et al.  Experiments on interfaces to support query expansion , 1997, J. Documentation.

[19]  Barry Smyth,et al.  On the Importance of Being Diverse: Analysing Similarity and Diversity in Web Search , 2004, Intelligent Information Processing.

[20]  Luo Si,et al.  Distributed information retrieval with skewed database size distributions , 2003 .