Session search modeling by partially observable Markov decision process

Session search, the task of document retrieval for a series of queries in a session, has been receiving increasing attention from the information retrieval research community. Session search exhibits the properties of rich user-system interactions and temporal dependency. These properties lead to our proposal of using partially observable Markov decision process to model session search. On the basis of a design choice schema for states, actions and rewards, we evaluate different combinations of these choices over the TREC 2012 and 2013 session track datasets. According to the experimental results, practical design recommendations for using PODMP in session search are discussed.

[1]  Gerard Salton,et al.  Improving retrieval performance by relevance feedback , 1997, J. Am. Soc. Inf. Sci..

[2]  Thorsten Joachims,et al.  A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization , 1997, ICML.

[3]  Ben Carterette,et al.  Overview of the TREC 2013 Session Track , 2013, TREC.

[4]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[5]  John D. Lafferty,et al.  Two-stage language models for information retrieval , 2002, SIGIR '02.

[6]  M. Bartlett,et al.  Markov Processes and Potential Theory , 1972, The Mathematical Gazette.

[7]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[8]  Grace Hui Yang,et al.  Win-win search: dual-agent stochastic game in session search , 2014, SIGIR.

[9]  Ryen W. White,et al.  Struggling and Success in Web Search , 2015, CIKM.

[10]  W. Bruce Croft,et al.  Iterative Search using Query Aspects , 2016, CIKM.

[11]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[12]  Ryen W. White,et al.  Modeling dwell time to predict click-level satisfaction , 2014, WSDM.

[13]  Chirag Shah,et al.  Extracting Information Seeking Intentions for Web Search Sessions , 2016, SIGIR.

[14]  Richard M. Schwartz,et al.  A hidden Markov model information retrieval system , 1999, SIGIR '99.

[15]  Grace Hui Yang,et al.  A POMDP model for content-free document re-ranking , 2014, SIGIR.

[16]  Ryen W. White,et al.  Modeling and analysis of cross-session search tasks , 2011, SIGIR.

[17]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs , 1978, Oper. Res..

[18]  Elaine Toms,et al.  Search Behaviour Before and After Search Success , 2016, SAL@SIGIR.

[19]  Bhaskar Mitra,et al.  Exploring Session Context using Distributed Representations of Queries and Reformulations , 2015, SIGIR.

[20]  Paul N. Bennett,et al.  Toward whole-session relevance: exploring intrinsic diversity in web search , 2013, SIGIR.

[21]  L. Baum,et al.  Statistical Inference for Probabilistic Functions of Finite State Markov Chains , 1966 .

[22]  Jun Wang,et al.  Sequential selection of correlated ads by POMDPs , 2012, CIKM.

[23]  Filip Radlinski,et al.  Learning diverse rankings with multi-armed bandits , 2008, ICML '08.

[24]  Steve Fox,et al.  Evaluating implicit measures to improve web search , 2005, TOIS.

[25]  ChengXiang Zhai,et al.  Implicit user modeling for personalized search , 2005, CIKM '05.

[26]  Charles L. A. Clarke,et al.  Efficient and effective spam filtering and re-ranking for large web datasets , 2010, Information Retrieval.

[27]  Ben Carterette,et al.  Overview of the TREC 2012 Session Track , 2012, TREC.

[28]  Lydia B. Chilton,et al.  Addressing people's information needs directly in a web search result page , 2011, WWW.

[29]  Grace Hui Yang,et al.  Utilizing query change for session search , 2013, SIGIR.

[30]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[31]  Michael N. Katehakis,et al.  The Multi-Armed Bandit Problem: Decomposition and Computation , 1987, Math. Oper. Res..

[32]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[33]  Jun Wang,et al.  Interactive exploratory search for multi page search results , 2013, WWW.

[34]  Katja Hofmann,et al.  Balancing Exploration and Exploitation in Learning to Rank Online , 2011, ECIR.

[35]  Moshe Tennenholtz,et al.  Ranking systems: the PageRank axioms , 2005, EC '05.

[36]  Ryen W. White,et al.  Struggling or exploring?: disambiguating long search sessions , 2014, WSDM.