Improving Session Search Performance with a Multi-MDP Model

To fulfill some sophisticated information needs in Web search, users may submit multiple queries in a search session. Session search aims to provide an optimized document rank by utilizing query log as well as user interaction behaviors within a search session. Although a number of solutions were proposed to solve the session search problem, most of these efforts simply assume that users’ search intents stay unchanged during the search process. However, most complicated search tasks involve exploratory processes where users’ intents evolve while interacting with search results. The evolving process leaves the static search intent assumption unreasonable and hurts the performance of document rank. To shed light on this research question, we propose a system with multiple agents which adjusts its framework by a self-adaption mechanism. In the framework, each agent models the document ranking as a Markov Decision Process (MDP) and updates its parameters by Reinforcement Learning algorithms. Experimental results on TREC Session Track datasets (2013 & 2014) show the effectiveness of the proposed framework.

[1]  Yang Song,et al.  Optimal rare query suggestion with implicit user feedback , 2010, WWW '10.

[2]  Nicholas J. Belkin,et al.  Personalizing information retrieval for multi-session tasks: the roles of task stage and task type , 2010, SIGIR '10.

[3]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[4]  Grace Hui Yang,et al.  A POMDP model for content-free document re-ranking , 2014, SIGIR.

[5]  Jacek Gwizdka,et al.  Proceedings of the Second International Workshop on Search as Learning, SAL 2016, co-located with the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, July 21st, 2016 , 2016, SAL@SIGIR.

[6]  J. Pitman.  Combinatorial Stochastic Processes , 2006 .

[7]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[8]  Jiafeng Guo,et al.  Reinforcement Learning to Rank with Markov Decision Process , 2017, SIGIR.

[9]  Grace Hui Yang,et al.  Win-win search: dual-agent stochastic game in session search , 2014, SIGIR.

[10]  Ben Carterette,et al.  Overview of the TREC 2013 Session Track , 2013, TREC.

[11]  Ben Carterette,et al.  Simulating simple user behavior for system effectiveness evaluation , 2011, CIKM '11.

[12]  Ben Carterette,et al.  Overview of the TREC 2014 Session Track , 2014, TREC.

[13]  Jacek Gwizdka,et al.  Search as Learning (SAL) Workshop 2016 , 2016, SIGIR.

[14]  James Allan,et al.  Task-aware query recommendation , 2013, SIGIR.

[15]  Grace Hui Yang,et al.  Utilizing query change for session search , 2013, SIGIR.

[16]  Paul N. Bennett,et al.  Toward whole-session relevance: exploring intrinsic diversity in web search , 2013, SIGIR.

[17]  Filip Radlinski,et al.  Query chains: learning to rank from implicit feedback , 2005, KDD '05.

[18]  Susan T. Dumais,et al.  To personalize or not to personalize: modeling queries with variation in user intent , 2008, SIGIR '08.

[19]  Ryen W. White,et al.  Modeling and analysis of cross-session search tasks , 2011, SIGIR.

[20]  Thorsten Joachims,et al.  A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization , 1997, ICML.

[21]  Grace Hui Yang,et al.  Effective Structured Query Formulation for Session Search , 2012, TREC.

[22]  Wei Chu,et al.  Learning to extract cross-session search tasks , 2013, WWW.

[23]  Shuguang Han,et al.  PITT at TREC 2011 Session Track , 2011, TREC.