Proactive identification of query failure

When a user fails to find any useful information to support the task at hand after issuing a query, the user experiences a query failure. Since users possess limited cognitive resources, query failures often lead to user frustration as no clear benefit is obtained from the associated search interactions. Therefore, to improve users' search experiences, we conducted a controlled‐lab study with 40 participants, seeking to explore the extent to which query failures can be proactively identified before users start examining the retrieved results. Specifically, based on the data collected from 693 query segments generated in 80 search sessions, we used past search behaviors and current query attributes as features to build classifiers and examined the performance in capturing query failures. We report that (1) analytics algorithms utilizing past search behavioral data have significantly better performances than the baseline model in tasks of different types, and (2) The knowledge of users' search intentions can help improve the performance of the prediction model. Results pave way for developing proactive system supports for task‐based search interactions.

[1]  Chirag Shah,et al.  Exploring the immediate and short-term effects of peer advice and cognitive authority on Web search behavior , 2019, Inf. Process. Manag..

[2]  Falk Scholer,et al.  The effect of threshold priming and need for cognition on relevance calibration and assessment , 2013, SIGIR.

[3]  Ryen W. White,et al.  Predicting query performance using query, result, and user interaction features , 2010, RIAO.

[4]  Robert G. Capra,et al.  The Effects of Manipulating Task Determinability on Search Behaviors and Outcomes , 2018, SIGIR.

[5]  Chirag Shah,et al.  Investigating the Impacts of Expectation Disconfirmation on Web Search , 2019, CHIIR.

[6]  Anne Aula,et al.  How does search behavior change as search becomes more difficult? , 2010, CHI.

[7]  Chirag Shah,et al.  Predicting Information Seeking Intentions from Search Behaviors , 2017, SIGIR.

[8]  Nicholas J. Belkin,et al.  A faceted approach to conceptualizing tasks in information seeking , 2008, Inf. Process. Manag..

[9]  Chirag Shah,et al.  Exploring the relationships between search intentions and query reformulations , 2016, ASIST.

[10]  Daqing He,et al.  Searching, browsing, and clicking in a search session: changes in user behavior by task and over time , 2014, SIGIR.

[11]  J. Liu,et al.  Usefulness as the Criterion for Evaluation of Interactive Information Retrieval , 2009 .

[12]  James Allan,et al.  Predicting searcher frustration , 2010, SIGIR.

[13]  Vitor R. Carvalho,et al.  Reducing long queries using query quality predictors , 2009, SIGIR.

[14]  Jacek Gwizdka,et al.  Search behaviors in different task types , 2010, JCDL '10.

[15]  Mark S. Ackerman,et al.  The perfect search engine is not enough: a study of orienteering behavior in directed search , 2004, CHI.

[16]  Iadh Ounis,et al.  Query performance prediction , 2006, Inf. Syst..

[17]  Jiqun Liu,et al.  Toward a unified model of human information behavior: an equilibrium perspective , 2017, J. Documentation.

[18]  Jaime Arguello,et al.  Development and Evaluation of Search Tasks for IIR Experiments using a Cognitive Complexity Framework , 2015, ICTIR.

[19]  Catherine L. Smith,et al.  User adaptation: good results from poor systems , 2008, SIGIR '08.

[20]  Chirag Shah,et al.  The Role of the Task Topic in Web Search of Different Task Types , 2018, CHIIR.

[21]  Peter Ingwersen,et al.  Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory , 1996, J. Documentation.

[22]  Chirag Shah,et al.  Information Fostering - Being Proactive with Information Seeking and Retrieval: Perspective Paper , 2018, CHIIR.

[23]  Chirag Shah,et al.  Search successes and failures in query segments and search tasks: A field study , 2017, ASIST.

[24]  Djoerd Hiemstra,et al.  A survey of pre-retrieval query performance predictors , 2008, CIKM '08.

[25]  Kun Huang,et al.  User evaluation of query quality , 2012, SIGIR '12.

[26]  Chirag Shah,et al.  User Activity Patterns During Information Search , 2015, ACM Trans. Inf. Syst..

[27]  Chirag Shah,et al.  Task, Information Seeking Intentions, and User Behavior: Toward A Multi-level Understanding of Web Search , 2019, CHIIR.

[28]  Dorota Glowacka,et al.  Interactive Intent Modeling for Exploratory Search , 2018, ACM Trans. Inf. Syst..