Modeling term dependencies with quantum language models for IR

Traditional information retrieval (IR) models use bag-of-words as the basic representation and assume that some form of independence holds between terms. Representing term dependencies and defining a scoring function capable of integrating such additional evidence is theoretically and practically challenging. Recently, Quantum Theory (QT) has been proposed as a possible, more general framework for IR. However, only a limited number of investigations have been made and the potential of QT has not been fully explored and tested. We develop a new, generalized Language Modeling approach for IR by adopting the probabilistic framework of QT. In particular, quantum probability could account for both single and compound terms at once without having to extend the term space artificially as in previous studies. This naturally allows us to avoid the weight-normalization problem, which arises in the current practice by mixing scores from matching compound terms and from matching single terms. Our model is the first practical application of quantum probability to show significant improvements over a robust bag-of-words baseline and achieves better performance on a stronger non bag-of-words baseline.

[1]  Gary Geunbae Lee,et al.  Dependency Structure Applied to Language Modeling for Information Retrieval , 2006 .

[2]  John C. Henderson,et al.  Direct Maximization of Average Precision by Hill-Climbing, with a Comparison to a Maximum Entropy Approach , 2004, HLT-NAACL.

[3]  James Allan,et al.  A comparison of statistical significance tests for information retrieval evaluation , 2007, CIKM '07.

[4]  W. Bruce Croft,et al.  Modeling higher-order term dependencies in information retrieval using query hypergraphs , 2012, SIGIR '12.

[5]  ChengXiang Zhai,et al.  Positional language models for information retrieval , 2009, SIGIR.

[6]  W. Bruce Croft,et al.  A general language model for information retrieval , 1999, CIKM '99.

[7]  Dawei Song,et al.  A Novel Re-ranking Approach Inspired by Quantum Measurement , 2011, ECIR.

[8]  W. Bruce Croft,et al.  A general language model for information retrieval (poster abstract) , 1999, SIGIR '99.

[9]  Jinglei Zhao,et al.  A proximity language model for information retrieval , 2009, SIGIR.

[10]  Massimo Melucci,et al.  Deriving a Quantum Information Retrieval Basis , 2013, Comput. J..

[11]  W. Bruce Croft,et al.  A quasi-synchronous dependence model for information retrieval , 2011, CIKM '11.

[12]  Matthew Lease An improved markov random field model for supporting verbose queries , 2009, SIGIR.

[13]  M. Pretti A message-passing algorithm with damping , 2005 .

[14]  C. J. van Rijsbergen,et al.  Quantum Mechanics and Information Retrieval , 2011, Advanced Topics in Information Retrieval.

[15]  Jianfeng Gao,et al.  Dependence language model for information retrieval , 2004, SIGIR '04.

[16]  W. Bruce Croft,et al.  A Markov random field model for term dependencies , 2005, SIGIR '05.

[17]  C. J. van Rijsbergen,et al.  The geometry of information retrieval , 2004 .

[18]  Guido Zuccon,et al.  Using the Quantum Probability Ranking Principle to Rank Interdependent Documents , 2010, ECIR.

[19]  Rohini K. Srihari,et al.  Biterm language models for document retrieval , 2002, SIGIR '02.

[20]  A. Gleason Measures on the Closed Subspaces of a Hilbert Space , 1957 .

[21]  Clement T. Yu,et al.  A theory of term importance in automatic text analysis , 1974, J. Am. Soc. Inf. Sci..

[22]  W. Bruce Croft,et al.  Indri at TREC 2005: Terabyte Track , 2005, TREC.

[23]  Joel L. Fagan,et al.  Automatic phrase indexing for document retrieval , 1987, SIGIR '87.

[24]  W. Bruce Croft,et al.  Linear feature-based models for information retrieval , 2007, Information Retrieval.

[25]  E. Knill,et al.  Diluted maximum-likelihood algorithm for quantum tomography , 2006, quant-ph/0611244.

[26]  R. Blume-Kohout Optimal, reliable estimation of quantum states , 2006, quant-ph/0611080.

[27]  Olivier Chapelle,et al.  Expected reciprocal rank for graded relevance , 2009, CIKM.

[28]  Xin Li,et al.  Investigation of partial query proximity in web search , 2008, WWW.

[29]  ChengXiang Zhai,et al.  Statistical Language Models for Information Retrieval: A Critical Review , 2008, Found. Trends Inf. Retr..

[30]  H. Umegaki CONDITIONAL EXPECTATION IN AN OPERATOR ALGEBRA, II , 1954 .

[31]  A. I. Lvovsky,et al.  Iterative maximum-likelihood reconstruction in quantum homodyne tomography , 2003, quant-ph/0311097.

[32]  R. Blume-Kohout Hedged maximum likelihood quantum state estimation. , 2010, Physical review letters.

[33]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[34]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[35]  Manfred K. Warmuth,et al.  Bayesian generalized probability calculus for density matrices , 2009, Machine Learning.

[36]  C. J. van Rijsbergen,et al.  What can quantum theory bring to information retrieval , 2010, CIKM.

[37]  W. Bruce Croft,et al.  Parameterized concept weighting in verbose queries , 2011, SIGIR.

[38]  Claire Cardie,et al.  An Analysis of Statistical and Syntactic Phrases , 1997, RIAO.

[39]  Stephen E. Robertson,et al.  A probabilistic model of information retrieval: development and comparative experiments - Part 2 , 2000, Inf. Process. Manag..