CNM: An Interpretable Complex-valued Network for Matching

This paper seeks to model human language by the mathematical framework of quantum physics. With the well-designed mathematical formulations in quantum physics, this framework unifies different linguistic units in a single complex-valued vector space, e.g. words as particles in quantum states and sentences as mixed systems. A complex-valued network is built to implement this framework for semantic matching. With well-constrained complex-valued components, the network admits interpretations to explicit physical meanings. The proposed complex-valued network for matching (CNM) achieves comparable performances to strong CNN and RNN baselines on two benchmarking question answering (QA) datasets.

[1]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[2]  A. I. Lvovsky,et al.  Iterative maximum-likelihood reconstruction in quantum homodyne tomography , 2003, quant-ph/0311097.

[3]  Jimmy J. Lin,et al.  Multi-Perspective Sentence Similarity Modeling with Convolutional Neural Networks , 2015, EMNLP.

[4]  Lei Yu,et al.  Deep Learning for Answer Sentence Selection , 2014, ArXiv.

[5]  Xuanjing Huang,et al.  Convolutional Neural Tensor Network Architecture for Community-Based Question Answering , 2015, IJCAI.

[6]  Bowen Zhou,et al.  LSTM-based Deep Learning Models for non-factoid answer selection , 2015, ArXiv.

[7]  Yi Yang,et al.  WikiQA: A Challenge Dataset for Open-Domain Question Answering , 2015, EMNLP.

[8]  Jimmy J. Lin,et al.  Pairwise Word Interaction Modeling with Deep Neural Networks for Semantic Similarity Measurement , 2016, NAACL.

[9]  R. Hughes,et al.  The Structure and Interpretation of Quantum Mechanics , 1989 .

[10]  Anna Wierzbicka,et al.  Semantic and lexical universals : theory and empirical findings , 1994 .

[11]  Yoshua Bengio,et al.  Modeling term dependencies with quantum language models for IR , 2013, SIGIR.

[12]  Dawei Song,et al.  A Quantum Many-body Wave Function Inspired Language Modeling Approach , 2018, CIKM.

[13]  Dawei Song,et al.  End-to-End Quantum-like Language Models with Application to Question Answering , 2018, AAAI.

[14]  Peter Bruza,et al.  Entangling words and meaning , 2008 .

[15]  Sandro Sozzo,et al.  A Quantum Probability Explanation in Fock Space for Borderline Contradictions , 2013, 1311.6050.

[16]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[17]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[18]  M. Ježek,et al.  Iterative algorithm for reconstruction of entangled states , 2000, quant-ph/0009093.

[19]  W. Bruce Croft,et al.  aNMM: Ranking Short Answer Texts with Attention-Based Neural Matching Model , 2016, CIKM.

[20]  Bowen Zhou,et al.  Attentive Pooling Networks , 2016, ArXiv.

[21]  Phil Blunsom,et al.  Neural Variational Inference for Text Processing , 2015, ICML.

[22]  Ellen M. Voorhees,et al.  Building a question answering test collection , 2000, SIGIR '00.

[23]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[24]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[25]  A. Gleason.  Measures on the Closed Subspaces of a Hilbert Space , 1957 .

[26]  C. J. van Rijsbergen,et al.  The geometry of information retrieval , 2004 .

[27]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[28]  Di Wang,et al.  A Long Short-Term Memory Model for Answer Sentence Selection in Question Answering , 2015, ACL.

[29]  Zachary Chase Lipton.  The mythos of model interpretability , 2016, ACM Queue.

[30]  Diederik Aerts,et al.  Quantum Entanglement in Concept Combinations , 2013, ArXiv.