A Quantum-Theoretic Approach to Distributional Semantics

In this paper we explore the potential of quantum theory as a formal framework for capturing lexical meaning. We present a novel semantic space model that is syntactically aware, takes word order into account, and features key quantum aspects such as superposition and entanglement. We define a dependency-based Hilbert space and show how to represent the meaning of words by density matrices that encode dependency neighborhoods. Experiments on word similarity and association reveal that our model achieves results competitive with a variety of classical models.

[1]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[2]  Joseph C. H. Chen,et al.  Quantum computation and natural language processing , 2002 .

[3]  Carina Silberer,et al.  Grounded Models of Semantic Representation , 2012, EMNLP.

[4]  Diederik Aerts,et al.  Quantum Structure in Cognition , 2008, 0805.3850.

[5]  Alessandro Lenci,et al.  Distributional Memory: A General Framework for Corpus-Based Semantics , 2010, CL.

[6]  B. Baaquie.  Quantum Finance: Path Integrals and Hamiltonians for Options and Interest Rates , 2004 .

[7]  Peter Bruza,et al.  Entangling words and meaning , 2008 .

[8]  Gregory Grefenstette,et al.  Explorations in automatic thesaurus discovery , 1994 .

[9]  Andrew Y. Ng,et al.  Improving Word Representations via Global Context and Multiple Word Prototypes , 2012, ACL.

[10]  Vlatko Vedral,et al.  Introduction to Quantum Information Science , 2006 .

[11]  Diederik Aerts,et al.  LETTER TO THE EDITOR: Quantum aspects of semantic analysis and symbolic artificial intelligence , 2003, quant-ph/0309022.

[12]  Edward E. Smith,et al.  On the adequacy of prototype theory as a theory of concepts , 1981, Cognition.

[13]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[14]  A. Tversky,et al.  The Disjunction Effect in Choice under Uncertainty , 1992 .

[15]  Peter Bruza,et al.  Is There Something Quantum-Like about the Human Mental Lexicon? , 2009, INEX.

[16]  Kirsty Kitto,et al.  Is there something quantum-like about the human mental lexicon? , 2009 .

[17]  G. Denhière,et al.  A Computational Model of Children's Semantic Memory , 2004 .

[18]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[19]  D. Kleppner,et al.  One Hundred Years of Quantum Physics , 2000, Science.

[20]  Thomas A. Schreiber,et al.  The University of South Florida free association, rhyme, and word fragment norms , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[21]  Peter Bruza,et al.  Quantum Logic of Semantic Space: An Exploratory Investigation of Context Effects in Practical Reasoning , 2005, We Will Show Them!.

[22]  Jerrold L. Aronson.  The Structure and Interpretation of Quantum Mechanics , 1992 .

[23]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[24]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[25]  Paul Adrien Maurice Dirac,et al.  A new notation for quantum mechanics , 1939, Mathematical Proceedings of the Cambridge Philosophical Society.

[26]  Ehud Rivlin,et al.  Placing search in context: the concept revisited , 2002, TOIS.

[27]  Stephen Clark,et al.  Concrete Sentence Spaces for Compositional Distributional Models of Meaning , 2010, IWCS.

[28]  Jerome R Busemeyer,et al.  Can quantum probability provide a new direction for cognitive modeling? , 2013, The Behavioral and brain sciences.

[29]  Yoshua Bengio,et al.  Neural net language models , 2008, Scholarpedia.

[30]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[31]  Mirella Lapata,et al.  A Comparison of Vector-based Representations for Semantic Composition , 2012, EMNLP.

[32]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[33]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[34]  Christopher D. Manning,et al.  The Stanford Typed Dependencies Representation , 2008, CF+CDPE@COLING.

[35]  Jerome R. Busemeyer,et al.  Quantum Models of Cognition and Decision , 2012 .

[36]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[37]  Andrei Khrennikov,et al.  Ubiquitous Quantum Structure: From Psychology to Finance , 2010 .

[38]  Dekang Lin,et al.  Automatic Retrieval and Clustering of Similar Words , 1998, ACL.