Interactive spoken content retrieval by extended query model and continuous state space Markov Decision Process

Interactive retrieval is important for spoken content because the retrieved spoken items are not only difficult to be shown on the screen but also scanned and selected by the user, in addition to the speech recognition uncertainty. The user cannot playback and go through all the retrieved items to find out what he is looking for. Markov Decision Process (MDP) was used in a previous work to help the system take different actions to interact with the user based on an estimated retrieval performance, but the MDP state was represented by the less precise quantized retrieval performance metric. In this paper, we consider the retrieval performance metric as a continuous state variable in MDP and optimize the MDP by fitted value iteration (FVI).We also use query expansion with the language modeling retrieval framework to produce the next set of retrieval results. Improved performance was found in the preliminary experiments.

[1]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.

[2]  Stuart Dreyfus,et al.  Richard Bellman on the Birth of Dynamic Programming , 2002, Oper. Res..

[3]  David Robins,et al.  Interactive Information Retrieval: Context and Basic Notions , 2000, Informing Sci. Int. J. an Emerg. Transdiscipl..

[4]  Lin-Shan Lee,et al.  Improved semantic retrieval of spoken content by language models enhanced with acoustic similarity graph , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[5]  Csaba Szepesvári,et al.  Finite-Time Bounds for Fitted Value Iteration , 2008, J. Mach. Learn. Res..

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[8]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[9]  Michael K. Buckland,et al.  Annual Review of Information Science and Technology , 2006, J. Documentation.

[10]  Tatsuya Kawahara,et al.  Bayes Risk-based Dialogue Management for Document Retrieval System with Speech Interface , 2008, COLING.

[11]  W. Bruce Croft,et al.  Predicting query performance , 2002, SIGIR '02.

[12]  ChengXiang Zhai,et al.  Improving retrieval accuracy of difficult queries through generalizing negative document language models , 2011, CIKM '11.

[13]  Vipin Kumar,et al.  Introduction to Data Mining, (First Edition) , 2005 .

[14]  Iadh Ounis,et al.  Query performance prediction , 2006, Inf. Syst..

[15]  R. Bellman.  A Markovian Decision Process , 1957 .

[16]  Csaba Szepesvári,et al.  Fitted Q-iteration in continuous action-space MDPs , 2007, NIPS.

[17]  Tao Tao,et al.  Regularized estimation of mixture models for robust pseudo-relevance feedback , 2006, SIGIR.

[18]  James R. Glass,et al.  Automating Crowd-supervised Learning for Spoken Language Systems , 2012, INTERSPEECH.

[19]  Hwee Tou Ng,et al.  Statistical lattice-based spoken document retrieval , 2010, TOIS.

[20]  Matthieu Geist,et al.  Optimizing spoken dialogue management with fitted value iteration , 2010, INTERSPEECH.

[21]  Lin-Shan Lee,et al.  Interactive Spoken Document Retrieval With Suggested Key Terms Ranked by a Markov Decision Process , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[22]  O. Pietquin,et al.  Optimizing Spoken Dialogue Management from Data Corpora with Fitted Value Iteration , 2010 .

[23]  ChengXiang Zhai,et al.  A study of methods for negative relevance feedback , 2008, SIGIR '08.

[24]  Shie Mannor,et al.  Regularized Fitted Q-iteration: Application to Planning , 2008, EWRL.

[25]  Thomas L. Griffiths,et al.  The Author-Topic Model for Authors and Documents , 2004, UAI.

[26]  L. J. Comrie,et al.  Mathematical Tables and Other Aids to Computation. , 1946 .

[27]  Falk Scholer,et al.  Effective Pre-retrieval Query Performance Prediction Using Similarity and Variability Evidence , 2008, ECIR.

[28]  Thomas Hofmann,et al.  Probabilistic latent semantic indexing , 1999, SIGIR '99.

[29]  Tatsuya Kawahara,et al.  Speech-Based Interactive Information Guidance System using Question-Answering Technique , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[30]  Csaba Szepesvári,et al.  Finite time bounds for sampling based fitted value iteration , 2005, ICML.

[31]  Sebastian Thrun,et al.  Issues in Using Function Approximation for Reinforcement Learning , 1999 .

[32]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[33]  Lin-Shan Lee,et al.  Type-II dialogue systems for information access from unstructured knowledge sources , 2007, 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU).

[34]  Lin-Shan Lee,et al.  Interactive Spoken Content Retrieval with Different Types of Actions Optimized By a Markov Decision Process , 2012, INTERSPEECH.

[35]  Shie Mannor,et al.  Regularized Fitted Q-Iteration for planning in continuous-space Markovian decision problems , 2009, 2009 American Control Conference.

[36]  W. Bruce Croft,et al.  Query performance prediction in web search environments , 2007, SIGIR.

[37]  James R. Glass,et al.  A Conversational Movie Search System Based on Conditional Random Fields , 2012, INTERSPEECH.

[38]  Ian Ruthven,et al.  Interactive information retrieval , 2008 .

[39]  John D. Lafferty,et al.  Model-based feedback in the language modeling approach to information retrieval , 2001, CIKM '01.