Dynamic Query Modeling for Related Content Finding

While watching television, people increasingly consume additional content related to what they are watching. We consider the task of finding video content related to a live television broadcast for which we leverage the textual stream of subtitles associated with the broadcast. We model this task as a Markov decision process and propose a method that uses reinforcement learning to directly optimize the retrieval effectiveness of queries generated from the stream of subtitles. Our dynamic query modeling approach significantly outperforms state-of-the-art baselines for stationary query modeling and for text-based retrieval in a television setting. In particular we find that carefully weighting terms and decaying these weights based on recency significantly improves effectiveness. Moreover, our method is highly efficient and can be used in a live television setting, i.e., in near real time.

[1]  Eugene A. Feinberg,et al.  Handbook of Markov Decision Processes , 2002 .

[2]  Vitor R. Carvalho,et al.  Reducing long queries using query quality predictors , 2009, SIGIR.

[3]  Filip Radlinski,et al.  How does clickthrough data reflect retrieval quality? , 2008, CIKM '08.

[4]  James Allan,et al.  Regression Rank: Learning to Meet the Opportunity of Descriptive Queries , 2009, ECIR.

[5]  Grace Hui Yang,et al.  Designing States, Actions, and Rewards for Using POMDP in Session Search , 2015, ECIR.

[6]  Katja Hofmann,et al.  Information Retrieval manuscript No. (will be inserted by the editor) Balancing Exploration and Exploitation in Listwise and Pairwise Online Learning to Rank for Information Retrieval , 2022 .

[7]  Roi Blanco,et al.  IntoNews: Online news retrieval using closed captions , 2015, Inf. Process. Manag..

[8]  Grace Hui Yang,et al.  Utilizing query change for session search , 2013, SIGIR.

[9]  James W. Cooper,et al.  Towards speech as a knowledge resource , 2001, CIKM '01.

[10]  Jun Wang,et al.  Interactive exploratory search for multi page search results , 2013, WWW.

[11]  W. Bruce Croft,et al.  Automatic suggestion of phrasal-concept queries for literature search , 2014, Inf. Process. Manag..

[12]  Miles Efron,et al.  Linear time series models for term weighting in information retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[13]  Martha Larson,et al.  Overview of VideoCLEF 2009: New Perspectives on Speech-based Multimedia Content Enrichment , 2009, CLEF.

[14]  Thorsten Joachims,et al.  Interactively optimizing information retrieval systems as a dueling bandits problem , 2009, ICML '09.

[15]  Monika Henzinger,et al.  Query-Free News Search , 2003, WWW '03.

[16]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[17]  M. de Rijke,et al.  Cognitive Temporal Document Priors , 2013, DIR.

[18]  Yiming Yang,et al.  Topic Detection and Tracking Pilot Study Final Report , 1998 .

[19]  Vanja Josifovski,et al.  Up next: retrieval methods for large scale related video suggestion , 2014, KDD.

[20]  M. de Rijke,et al.  Linking Archives Using Document Enrichment and Term Selection , 2011, TPDL.

[21]  Richard S. Sutton,et al.  Associative search network: A reinforcement learning associative memory , 1981, Biological Cybernetics.

[22]  Niranjan Balasubramanian,et al.  Exploring reductions for long web queries , 2010, SIGIR.

[23]  W. Bruce Croft,et al.  Parameterized concept weighting in verbose queries , 2011, SIGIR.

[24]  Matthew Lease.  An improved markov random field model for supporting verbose queries , 2009, SIGIR.

[25]  Kenneth Ward Church,et al.  Poisson mixtures , 1995, Natural Language Engineering.

[26]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[27]  Grace Hui Yang,et al.  Win-win search: dual-agent stochastic game in session search , 2014, SIGIR.

[28]  M. de Rijke,et al.  Adaptive Temporal Query Modeling , 2012, ECIR.

[29]  Marti A. Hearst.  Text Tiling: Segmenting Text into Multi-paragraph Subtopic Passages , 1997, CL.

[30]  Seong-Bae Park,et al.  A just-in-time keyword extraction from meeting transcripts using temporal and participant information , 2015, Journal of Intelligent Information Systems.

[31]  W. Bruce Croft,et al.  Discovering key concepts in verbose queries , 2008, SIGIR '08.

[32]  Maarten de Rijke,et al.  Feeding the Second Screen: Semantic Linking based on Subtitles , 2013, DIR.

[33]  W. Bruce Croft,et al.  Combining the language model and inference network approaches to retrieval , 2004, Inf. Process. Manag..

[34]  W. Bruce Croft,et al.  Generating queries from user-selected text , 2012, IIiX.

[35]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[36]  Ben Carterette,et al.  Evaluating Search Engines by Modeling the Relationship Between Relevance and Clicks , 2007, NIPS.

[37]  UP NEXT , 2006, Science.

[38]  R. Bellman.  A Markovian Decision Process , 1957 .