Dynamic Search - Optimizing the Game of Information Seeking

This article presents the emerging topic of dynamic search (DS). To position dynamic search in a larger research landscape, the article discusses in detail its relationship to related research topics and disciplines. The article reviews approaches to modeling dynamics during information seeking, with an emphasis on Reinforcement Learning (RL)-enabled methods. Details are given for how different approaches are used to model interactions among the human user, the search system, and the environment. The paper ends with a review of evaluations of dynamic search systems.

[1]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.

[2]  M. de Rijke,et al.  A Click Sequence Model for Web Search , 2018, SIGIR.

[3]  Ellen M. Voorhees,et al.  Overview of the TREC 2004 Robust Track. , 2004 .

[4]  Grace Hui Yang,et al.  Utilizing query change for session search , 2013, SIGIR.

[5]  Falk Scholer,et al.  RMIT @ TREC 2016 Dynamic Domain Track: Exploiting Passage Representation for Retrieval and Relevance Feedback , 2016, TREC.

[6]  Wenfei Liu,et al.  DUTIR at the Session Track in TREC 2011 , 2011, TREC.

[7]  Jiliang Tang,et al.  A Survey on Dialogue Systems: Recent Advances and New Frontiers , 2017, SKDD.

[8]  Huazheng Wang,et al.  Factorization Bandits for Interactive Recommendation , 2017, AAAI.

[9]  Yiming Yang,et al.  Modeling Expected Utility of Multi-session Information Distillation , 2009, ICTIR.

[10]  Umut Ozertem,et al.  Suggestion set utility maximization using session logs , 2011, CIKM '11.

[11]  Wei Lin,et al.  Transfer Learning for Context-Aware Question Matching in Information-seeking Conversations in E-commerce , 2018, ACL.

[12]  Jun Wang,et al.  Dynamic Information Retrieval Modeling , 2015, Synthesis Lectures on Information Concepts, Retrieval, and Services.

[13]  Rui Zhang,et al.  A Unified Processing Paradigm for Interactive Location-based Web Search , 2018, WSDM.

[14]  Ryen W. White,et al.  Supporting Exploratory Search, Introduction, Special Issue, Communications of the ACM , 2006 .

[15]  Grace Hui Yang,et al.  Overview of the CLEF Dynamic Search Evaluation Lab 2018 , 2018, CLEF.

[16]  Rajarshi Das,et al.  Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering , 2019, ICLR.

[17]  Kyunghyun Cho,et al.  Task-Oriented Query Reformulation with Reinforcement Learning , 2017, EMNLP.

[18]  Mark Levene,et al.  Search Engines: Information Retrieval in Practice , 2011, Comput. J..

[19]  Udo Kruschwitz,et al.  University of Essex at the TREC 2010 Session Track , 2010, TREC.

[20]  Tetsuya Sakai,et al.  Summaries, ranked retrieval and sessions: a unified framework for information access evaluation , 2013, SIGIR.

[21]  Xiaojun Yuan,et al.  U. Albany & USC at the TREC 2012 Session Track , 2012, TREC.

[22]  Scott B. Huffman,et al.  How well does result relevance predict session satisfaction? , 2007, SIGIR.

[23]  W. Bruce Croft,et al.  Search Engines - Information Retrieval in Practice , 2009 .

[24]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[25]  Doug Downey,et al.  Explanatory semantic relatedness and explicit spatialization for exploratory search , 2012, SIGIR '12.

[26]  Nicholas J. Belkin,et al.  Rutgers at the TREC 2012 Session Track , 2012, TREC.

[27]  Alessandro Bozzon,et al.  Liquid query: multi-domain exploratory search on the web , 2010, WWW '10.

[28]  W. Bruce Croft,et al.  Relevance-Based Language Models , 2001, SIGIR '01.

[29]  Luc Lamontagne,et al.  Laval University and Lakehead University at TREC Dynamic Domain 2015: Combination of Techniques for Subtopics Coverage , 2015, TREC.

[30]  Jade Goldstein-Stewart,et al.  The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries , 1998, SIGIR Forum.

[31]  Aaron Halfaker,et al.  User Session Identification Based on Strong Regularities in Inter-activity Time , 2014, WWW.

[32]  Henry Feild.  Endicott College at 2014 TREC Session Track , 2014, TREC.

[33]  Charles L. A. Clarke,et al.  Novelty and diversity in information retrieval evaluation , 2008, SIGIR '08.

[34]  Jun Wang,et al.  Interactive exploratory search for multi page search results , 2013, WWW.

[35]  Grace Hui Yang,et al.  Learning to Reinforce Search Effectiveness , 2015, ICTIR.

[36]  Grace Hui Yang,et al.  The water filling model and the cube test: multi-dimensional evaluation for professional search , 2013, CIKM.

[37]  Lovekesh Vig,et al.  Automatic Conversational Helpdesk Solution using Seq2Seq and Slot-filling Models , 2018, CIKM.

[38]  Edward Lank,et al.  Improving Exploratory Search Experience through Hierarchical Knowledge Graphs , 2017, SIGIR.

[39]  Runze Li,et al.  BUPT_PRIS at TREC 2012 Session Track , 2012, TREC.

[40]  Jianfeng Gao,et al.  A Neural Network Approach to Context-Sensitive Generation of Conversational Responses , 2015, NAACL.

[41]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[42]  Chirag Shah,et al.  Algorithmic mediation for collaborative exploratory search , 2008, SIGIR '08.

[43]  Rosie Jones,et al.  Beyond the session timeout: automatic hierarchical segmentation of search topics in query logs , 2008, CIKM '08.

[44]  Dongyan Zhao,et al.  How to Make Context More Useful? An Empirical Study on Context-Aware Neural Conversational Models , 2017, ACL.

[45]  Jianfeng Gao,et al.  A Persona-Based Neural Conversation Model , 2016, ACL.

[46]  Roberto Cornacchia,et al.  CWI at TREC 2011: Session, Web, and Medical , 2011, TREC.

[47]  Marcia J. Bates,et al.  The design of browsing and berrypicking techniques for the online search interface , 1989 .

[48]  Nicholas Jing Yuan,et al.  DRN: A Deep Reinforcement Learning Framework for News Recommendation , 2018, WWW.

[49]  Wei Zeng,et al.  From Greedy Selection to Exploratory Decision-Making: Diverse Ranking with Policy-Value Networks , 2018, SIGIR.

[50]  Yiqun Liu,et al.  Towards Designing Better Session Search Evaluation Metrics , 2018, SIGIR.

[51]  Ben Carterette,et al.  University of Delaware at TREC 2014 , 2014, TREC.

[52]  Hugo Zaragoza,et al.  The Probabilistic Relevance Framework: BM25 and Beyond , 2009, Found. Trends Inf. Retr..

[53]  Gary Marchionini,et al.  Information Seeking in Electronic Environments , 1995 .

[54]  Gary Marchionini,et al.  Exploratory search , 2006, Commun. ACM.

[55]  Yue Liu,et al.  ICTNET at Session Track TREC 2013 , 2011, TREC.

[56]  Massimo Melucci,et al.  Evaluation of a Feedback Algorithm inspired by Quantum Detection for Dynamic Search Tasks , 2016, TREC.

[57]  Grace Hui Yang,et al.  Designing States, Actions, and Rewards for Using POMDP in Session Search , 2015, ECIR.

[58]  Nicholas J. Belkin,et al.  Personalization of search results using interaction behaviors in search sessions , 2012, SIGIR '12.

[59]  Grace Hui Yang,et al.  TREC 2016 Dynamic Domain Track Overview , 2016, TREC.

[60]  Stephen E. Robertson,et al.  Relevance weighting of search terms , 1976, J. Am. Soc. Inf. Sci..

[61]  Katja Hofmann,et al.  The University of Amsterdam at TREC 2010: Session, Entity and Relevance Feedback , 2010, TREC.

[62]  Tie-Yan Liu,et al.  Learning to Rank for Information Retrieval , 2011 .

[63]  Ben Carterette,et al.  Implicit Feedback and Document Filtering for Retrieval Over Query Sessions , 2011, TREC.

[64]  Udo Kruschwitz,et al.  ’ s repository of research publications and other research outputs RGU-ISTI-Essex at TREC 2011 Session Track Conference or Workshop Item , 2018 .

[65]  Stephane Ross,et al.  Interactive Learning for Sequential Decisions and Predictions , 2013 .

[66]  Grace Hui Yang,et al.  Win-win search: dual-agent stochastic game in session search , 2014, SIGIR.

[67]  Xu Chen,et al.  Towards Conversational Search and Recommendation: System Ask, User Respond , 2018, CIKM.

[68]  Craig MacDonald,et al.  University of Glasgow at TREC 2015: Experiments with Terrier in Contextual Suggestion, Temporal Summarisation and Dynamic Domain Tracks , 2015, TREC.

[69]  Lois M. L. Delcambre,et al.  Discounted Cumulated Gain Based Evaluation of Multiple-Query IR Sessions , 2008, ECIR.

[70]  Wei Chu,et al.  A contextual-bandit approach to personalized news article recommendation , 2010, WWW '10.

[71]  Rui Yan,et al.  Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System , 2016, SIGIR.

[72]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[73]  Paul N. Bennett,et al.  Toward whole-session relevance: exploring intrinsic diversity in web search , 2013, SIGIR.

[74]  Dorota Glowacka,et al.  Balancing Exploration and Exploitation: Empirical Parameterization of Exploratory Search Systems , 2015, CIKM.

[75]  Leif Azzopardi,et al.  Modelling interaction with economic models of search , 2014, SIGIR.

[76]  Pieter Spronck,et al.  Monte-Carlo Tree Search: A New Framework for Game AI , 2008, AIIDE.

[77]  Ryen W. White,et al.  Exploratory Search: Beyond the Query-Response Paradigm , 2009, Exploratory Search: Beyond the Query-Response Paradigm.

[78]  Zhoujun Li,et al.  Sequential Match Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots , 2016, ArXiv.

[79]  Nick Craswell,et al.  Random walks on the click graph , 2007, SIGIR.

[80]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[81]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[82]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[83]  Gary Marchionini,et al.  Finding facts vs. browsing knowledge in hypertext systems , 1988, Computer.

[84]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[85]  Nir Levine,et al.  An Extended Relevance Model for Session Search , 2017, SIGIR.

[86]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[87]  Ben Carterette,et al.  Evaluating Retrieval over Sessions: The TREC Session Track 2011-2014 , 2016, SIGIR.

[88]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[89]  Jianfeng Gao,et al.  Deep Reinforcement Learning for Dialogue Generation , 2016, EMNLP.

[90]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[91]  Matthew Lease,et al.  Active learning to maximize accuracy vs. effort in interactive information retrieval , 2011, SIGIR.

[92]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[93]  Dorota Glowacka,et al.  Directing exploratory search with interactive intent modeling , 2013, CIKM.

[94]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[95]  Joseph Weizenbaum,et al.  and Machine , 1977 .

[96]  Charles L. A. Clarke,et al.  Time-based calibration of effectiveness measures , 2012, SIGIR '12.

[97]  Pernilla Qvarfordt,et al.  Looking ahead: query preview in exploratory search , 2013, SIGIR.

[98]  Grace Hui Yang,et al.  A Contextual Bandit Approach to Dynamic Search , 2017, ICTIR.

[99]  Michael K. Buckland,et al.  Information as thing , 1991, J. Am. Soc. Inf. Sci..

[100]  Ellen M. Voorhees,et al.  Overview of TREC 2003 , 2003, TREC.

[101]  Jianfeng Gao,et al.  Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access , 2016, ACL.

[102]  Lihong Li,et al.  Neural Approaches to Conversational AI , 2019, Found. Trends Inf. Retr..

[103]  R. Agrawal.  Sample mean based index policies by O(log n) regret for the multi-armed bandit problem , 1995, Advances in Applied Probability.

[104]  Jason Weston,et al.  Learning End-to-End Goal-Oriented Dialog , 2016, ICLR.

[105]  Jeff A. Bilmes,et al.  A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[106]  Hang Li,et al.  Neural Responding Machine for Short-Text Conversation , 2015, ACL.

[107]  Cheng Li,et al.  Multiple Queries as Bandit Arms , 2016, CIKM.

[108]  Anton Leuski,et al.  Relevance and reinforcement in interactive browsing , 2000, CIKM '00.

[109]  Danushka Bollegala,et al.  Collaborative exploratory search in real-world context , 2011, CIKM '11.

[110]  Xi Chen,et al.  Query Tracking for E-commerce Conversational Search: A Machine Comprehension Perspective , 2018, CIKM.

[111]  Emre Velipasaoglu,et al.  Intent-based diversification of web search results: metrics and algorithms , 2011, Information Retrieval.

[112]  Ian Ruthven,et al.  Interactive information retrieval , 2008 .

[113]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[114]  M. de Rijke,et al.  Attentive Memory Networks: Efficient Machine Reading for Conversational Search , 2017, ArXiv.

[115]  Rodrygo L. T. Santos,et al.  UFMG at the TREC 2016 Dynamic Domain track , 2016, TREC.

[116]  Yinan Zhang,et al.  A Sequential Decision Formulation of the Interface Card Model for Interactive IR , 2016, SIGIR.

[117]  Xuan Liu,et al.  Multi-view Response Selection for Human-Computer Conversation , 2016, EMNLP.

[118]  Yue Liu,et al.  ICTNET at Session Track TREC2014 , 2014, TREC.

[119]  David Hawking,et al.  Overview of the TREC-9 Web Track , 2000, TREC.

[120]  Shuguang Han,et al.  On Duplicate Results in a Search Session , 2012, TREC.

[121]  Filip Radlinski,et al.  A Theoretical Framework for Conversational Search , 2017, CHIIR.

[122]  John D. Lafferty,et al.  A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval , 2017, SIGF.

[123]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[124]  Enrique Alfonseca,et al.  Learning to Attend, Copy, and Generate for Session-Based Query Suggestion , 2017, CIKM.

[125]  Ed H. Chi,et al.  Top-K Off-Policy Correction for a REINFORCE Recommender System , 2018, WSDM.

[126]  Hang Li,et al.  An Information Retrieval Approach to Short Text Conversation , 2014, ArXiv.

[127]  Anne N. De Roeck,et al.  Autopoiesis, the immune system, and adaptive information filtering , 2009, Natural Computing.

[128]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..

[129]  Grace Hui Yang,et al.  Session Search by Direct Policy Learning , 2015, ICTIR.

[130]  Leif Azzopardi,et al.  The economics in interactive information retrieval , 2011, SIGIR.

[131]  Katja Hofmann,et al.  The University of Amsterdam at the TREC 2011 Session Track , 2011, TREC.

[132]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[133]  Artem Grotov,et al.  Online Learning to Rank for Information Retrieval: SIGIR 2016 Tutorial , 2016, SIGIR.

[134]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[135]  Jun Huang,et al.  Response Ranking with Deep Matching Networks and External Knowledge in Information-seeking Conversation Systems , 2018, SIGIR.

[136]  Pieter Abbeel,et al.  An Algorithmic Perspective on Imitation Learning , 2018, Found. Trends Robotics.

[137]  Roberto I. González-Ibáñez,et al.  Evaluating the synergic effect of collaboration in information seeking , 2011, SIGIR.

[138]  Wlodek Zadrozny,et al.  Toward an Interactive Patent Retrieval Framework based on Distributed Representations , 2018, SIGIR.

[139]  Matthias Hagen,et al.  Webis at TREC 2013-Session and Web Track , 2013, TREC.

[140]  Pia Borlund,et al.  The concept of relevance in IR , 2003, J. Assoc. Inf. Sci. Technol..

[141]  Fabrizio Silvestri,et al.  Identifying task-based sessions in search engine query logs , 2011, WSDM '11.

[142]  Bhaskar Mitra,et al.  Exploring Session Context using Distributed Representations of Queries and Reformulations , 2015, SIGIR.

[143]  Sergey Levine,et al.  Guided Policy Search , 2013, ICML.

[144]  Dorota Glowacka,et al.  How Consistent is Relevance Feedback in Exploratory Search? , 2018, CIKM.

[145]  Ludovic Denoyer,et al.  A Reinforcement Learning-driven Translation Model for Search-Oriented Conversational Systems , 2018, SCAI@EMNLP.

[146]  Wei Chu,et al.  Learning to extract cross-session search tasks , 2013, WWW.

[147]  Qifa Ke,et al.  Conversational Query Understanding Using Sequence to Sequence Modeling , 2018, WWW.

[148]  Grace Hui Yang,et al.  A Reinforcement Learning Approach for Dynamic Search , 2017, TREC.

[149]  Daqing He,et al.  Pitt at TREC 2013: Different Effects of Click-through and Past Queries on Whole-session Search Performance , 2013, TREC.

[150]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[151]  Christian Plaunt,et al.  On the Construction of Selection Systems. , 1994 .

[152]  Benjamin King,et al.  Cengage Learning at the TREC 2010 Session Track , 2010, TREC.