A Quantum Expectation Value Based Language Model with Application to Question Answering

Quantum-inspired language models have been introduced to Information Retrieval due to their transparency and interpretability. While exciting progresses have been made, current studies mainly investigate the relationship between density matrices of difference sentence subspaces of a semantic Hilbert space. The Hilbert space as a whole which has a unique density matrix is lack of exploration. In this paper, we propose a novel Quantum Expectation Value based Language Model (QEV-LM). A unique shared density matrix is constructed for the Semantic Hilbert Space. Words and sentences are viewed as different observables in this quantum model. Under this background, a matching score describing the similarity between a question-answer pair is naturally explained as the quantum expectation value of a joint question-answer observable. In addition to the theoretical soundness, experiment results on the TREC-QA and WIKIQA datasets demonstrate the computational efficiency of our proposed model with excellent performance and low time consumption.

[1]  D. A. Edwards.  The mathematical foundations of quantum mechanics , 1979, Synthese.

[2]  Fabio Tamburini,et al.  Towards Quantum Language Models , 2017, EMNLP.

[3]  C. J. van Rijsbergen,et al.  The geometry of information retrieval , 2004 .

[4]  C. J. van Rijsbergen,et al.  What can quantum theory bring to information retrieval , 2010, CIKM.

[5]  Bowen Zhou,et al.  Attentive Pooling Networks , 2016, ArXiv.

[6]  Yi Yang,et al.  WikiQA: A Challenge Dataset for Open-Domain Question Answering , 2015, EMNLP.

[7]  Phil Blunsom,et al.  Neural Variational Inference for Text Processing , 2015, ICML.

[8]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[9]  Dawei Song,et al.  End-to-End Quantum-like Language Models with Application to Question Answering , 2018, AAAI.

[10]  S. Semmes.  Topological Vector Spaces , 2003 .

[11]  C. J. van Rijsbergen,et al.  Quantum Mechanics and Information Retrieval , 2011, Advanced Topics in Information Retrieval.

[12]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[13]  R. Bhatia.  Positive Definite Matrices , 2007 .

[14]  Lei Yu,et al.  Deep Learning for Answer Sentence Selection , 2014, ArXiv.

[15]  J. J. Sakurai,et al.  Modern Quantum Mechanics , 1986 .

[16]  Dawei Song,et al.  Modeling Quantum Entanglements in Quantum Language Models , 2015, IJCAI.

[17]  Peter Bruza,et al.  Modelling Cued-Target Recall Using Quantum Inspired Models of Target Activation , 2015, QI.

[18]  Elham Kashefi,et al.  A Quantum-Theoretic Approach to Distributional Semantics , 2013, NAACL.

[19]  A. Gleason.  Measures on the Closed Subspaces of a Hilbert Space , 1957 .

[20]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[21]  Alessandro Moschitti,et al.  Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks , 2015, SIGIR.

[22]  Dawei Song,et al.  Exploration of Quantum Interference in Document Relevance Judgement Discrepancy , 2016, Entropy.

[23]  S. Lloyd,et al.  Quantum algorithms for supervised and unsupervised machine learning , 2013, 1307.0411.

[24]  C. J. van Rijsbergen,et al.  The Quantum Probability Ranking Principle for Information Retrieval , 2009, ICTIR.

[25]  Anna Wierzbicka,et al.  Semantic and lexical universals : theory and empirical findings , 1994 .

[26]  R. Hughes,et al.  The Structure and Interpretation of Quantum Mechanics , 1989 .

[27]  Dawei Song,et al.  A Quantum Many-body Wave Function Inspired Language Modeling Approach , 2018, CIKM.

[28]  Di Wang,et al.  A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering , 2015, ACL.

[29]  Yoshua Bengio,et al.  Modeling term dependencies with quantum language models for IR , 2013, SIGIR.

[30]  Massimo Melucci,et al.  CNM: An Interpretable Complex-valued Network for Matching , 2019, NAACL.

[31]  Alessandro Moschitti,et al.  Modeling Relational Information in Question-Answer Pairs with Convolutional Neural Networks , 2016, ArXiv.

[32]  Ellen M. Voorhees,et al.  Building a question answering test collection , 2000, SIGIR '00.

[33]  Jian-Yun Nie,et al.  Looking at Vector Space and Language Models for IR Using Density Matrices , 2013, QI.

[34]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[35]  Jun Wang,et al.  Automata Modeling for Cognitive Interference in Users' Relevance Judgment , 2010, AAAI Fall Symposium: Quantum Informatics for Cognitive, Social, and Semantic Processes.