Investigation of User Search Behavior While Facing Heterogeneous Search Services

With Web users' search tasks becoming increasingly complex, a single information source cannot necessarily satisfy their information needs. Searchers may rely on heterogeneous sources to complete their tasks, such as search engines, Community Question Answering (CQA), encyclopedia sites, and crowdsourcing platforms. Previous works focus on interaction behaviors with federated search results, including how to compose a federated Web search result page and what factors affect users' interaction behavior on aggregated search interfaces. However, little is known about which factors are crucial in determining users' search outcomes while facing multiple heterogeneous search services. In this paper, we design a lab-based user study to analyze what explicit and implicit factors affect search outcomes (information gain and user satisfaction) when users have access to heterogeneous information sources. In the study, each participant can access three different kinds of search services: a general search engine (Bing), a general CQA portal (Baidu Knows), and a high-quality CQA portal (Zhihu). Using questionnaires and interaction log data, we extract explicit and implicit signals to analyze how users' search outcomes are correlated with their behaviors on different information sources. Experimental results indicate that users' search experiences on CQA portals (such as users' perceived usefulness and number of result clicks) positively affect search outcome (information gain), while search satisfaction is significantly correlated with some other factors such as users' familiarity, interest and difficulty of the task. Besides, users' search satisfaction can be more accurately predicted by the implicit factors than search outcomes.

[1]  Eugene Agichtein,et al.  Predicting web search success with fine-grained interaction data , 2012, CIKM.

[2]  Fernando Diaz,et al.  Sources of evidence for vertical selection , 2009, SIGIR.

[3]  Yiqun Liu,et al.  Different Users, Different Opinions: Predicting Search Satisfaction with Mouse Movement Information , 2015, SIGIR.

[4]  Christian Bizer,et al.  Faceted Wikipedia Search , 2010, BIS.

[5]  Joemon M. Jose,et al.  Which vertical search engines are relevant? , 2013, WWW '13.

[6]  Barry Smyth,et al.  Capturing community search expertise for personalized web search using snippet-indexes , 2006, CIKM '06.

[7]  Salman Ahmad,et al.  Weblines: Enabling the Social Transfer of Web Search Expertise using User-Generated Short-form Timelines , 2011, CSCL.

[8]  Ryen W. White,et al.  Understanding and Predicting Graded Search Satisfaction , 2015, WSDM.

[9]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[10]  Huanbo Luan,et al.  Predicting Search User Examination with Visual Saliency , 2016, SIGIR.

[11]  Tetsuya Sakai,et al.  Evaluating Heterogeneous Information Access (Position Paper) , 2013 .

[12]  Christopher L. Tucci,et al.  Crowdsourcing As a Solution to Distant Search , 2012 .

[13]  Neema Moraveji.  User interface designs to support the social transfer of web search expertise , 2010, SIGIR '10.

[14]  Yang Song,et al.  Modeling action-level satisfaction for search task satisfaction prediction , 2014, SIGIR.

[15]  Joemon M. Jose,et al.  Evaluating aggregated search pages , 2012, SIGIR '12.

[16]  Jaime Arguello,et al.  Task complexity, vertical display and user interaction in aggregated search , 2012, SIGIR '12.

[17]  Dan Morris,et al.  Investigating the querying and browsing behavior of advanced search engine users , 2007, SIGIR.

[18]  Tapas Kanungo,et al.  On composition of a federated web search result page: using online users to provide pairwise preference for heterogeneous verticals , 2011, WSDM '11.

[19]  Kevyn Collins-Thompson,et al.  Assessing Learning Outcomes in Web Search: A Comparison of Tasks and Query Strategies , 2016, CHIIR.

[20]  Eugene Agichtein,et al.  Predicting information seeker satisfaction in community question answering , 2008, SIGIR '08.

[21]  Yan Zhang,et al.  Individual Differences and Online Health Information Source Selection , 2016, CHIIR.

[22]  Nick Craswell,et al.  Beyond clicks: query reformulation as a predictor of search satisfaction , 2013, CIKM.

[23]  Robert Villa,et al.  Factors affecting click-through behavior in aggregated search interfaces , 2010, CIKM.

[24]  Fernando Diaz,et al.  A Methodology for Evaluating Aggregated Search Results , 2011, ECIR.

[25]  Floor Sietsma,et al.  Evaluating intuitiveness of vertical-aware click models , 2014, SIGIR.