Open System Categorical Quantum Semantics in Natural Language Processing

Originally inspired by categorical quantum mechanics (Abramsky and Coecke, LiCS'04), the categorical compositional distributional model of natural language meaning of Coecke, Sadrzadeh and Clark provides a conceptually motivated procedure to compute the meaning of a sentence, given its grammatical structure within a Lambek pregroup and a vectorial representation of the meaning of its parts. The predictions of this first model have outperformed that of other models in mainstream empirical language processing tasks on large scale data. Moreover, just like CQM allows for varying the model in which we interpret quantum axioms, one can also vary the model in which we interpret word meaning. In this paper we show that further developments in categorical quantum mechanics are relevant to natural language processing too. Firstly, Selinger's CPM-construction allows for explicitly taking into account lexical ambiguity and distinguishing between the two inherently different notions of homonymy and polysemy. In terms of the model in which we interpret word meaning, this means a passage from the vector space model to density matrices. Despite this change of model, standard empirical methods for comparing meanings can be easily adopted, which we demonstrate by a small-scale experiment on real-world data. This experiment moreover provides preliminary evidence of the validity of our proposed new model for word meaning. Secondly, commutative classical structures as well as their non-commutative counterparts that arise in the image of the CPM-construction allow for encoding relative pronouns, verbs and adjectives, and finally, iteration of the CPM-construction, something that has no counterpart in the quantum realm, enables one to accommodate both entailment and ambiguity.

[1]  Stephen Clark,et al.  Mathematical Foundations for a Compositional Distributional Model of Meaning , 2010, ArXiv.

[2]  W. Marsden.  I and J , 2012 .

[3]  Kirsty Kitto,et al.  Is there something quantum-like about the human mental lexicon? , 2009 .

[4]  Mehrnoosh Sadrzadeh,et al.  Experimental Support for a Categorical Compositional Distributional Model of Meaning , 2011, EMNLP.

[5]  Robin Piedeleu,et al.  Ambiguity in Categorical Models of Meaning , 2014 .

[6]  E. Bach,et al.  An extension of classical transformational gram-mar , 1976 .

[7]  Stephen Clark,et al.  A quantum teleportation inspired algorithm produces sentence meaning from word meaning and grammatical structure , 2013, ArXiv.

[8]  Keye Martin,et al.  A Partial Order on Classical and Quantum States , 2010 .

[9]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[10]  Stephen Clark,et al.  The Frobenius anatomy of word meanings I: subject and object relative pronouns , 2013, J. Log. Comput..

[11]  D. Gorfein.  On the consequences of meaning selection: Perspectives on resolving lexical ambiguity. , 2001 .

[12]  A. Carboni,et al.  Cartesian bicategories I , 1987 .

[13]  Aleks Kissinger,et al.  Categories of quantum and classical channels , 2016, Quantum Inf. Process..

[14]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[15]  Stephen Pulman.  Compositional distributional semantics with compact closed categories and Frobenius algebras , 2014 .

[16]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[17]  Dimitri Kartsaklis,et al.  Compositional distributional semantics with compact closed categories and Frobenius algebras , 2015, ArXiv.

[18]  Richard Montague,et al.  ENGLISH AS A FORMAL LANGUAGE , 1975 .

[19]  Dimitri Kartsaklis,et al.  Reasoning about Meaning in Natural Language with Compact Closed Categories and Frobenius Algebras , 2014, ArXiv.

[20]  Anne Preller,et al.  Bell States and Negative Sentences in the Distributed Model of Meaning , 2011, Electron. Notes Theor. Comput. Sci..

[21]  Zellig S. Harris,et al.  Mathematical structures of language , 1968, Interscience tracts in pure and applied mathematics.

[22]  Peter Selinger,et al.  Dagger Compact Closed Categories and Completely Positive Maps: (Extended Abstract) , 2007, QPL.

[23]  Dusko Pavlovic,et al.  A new description of orthogonal bases , 2008, Mathematical Structures in Computer Science.

[24]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[25]  Stephen Clark,et al.  The Frobenius anatomy of word meanings II: possessive relative pronouns , 2014, J. Log. Comput..

[26]  Anne Preller,et al.  Free compact 2-categories , 2007, Mathematical Structures in Computer Science.

[27]  P. Selinger.  A Survey of Graphical Languages for Monoidal Categories , 2009, 0908.3347.

[28]  Robert W. Spekkens,et al.  Picturing classical and quantum Bayesian inference , 2011, Synthese.

[29]  Esma Balkr,et al.  Using Density Matrices in a Compositional Distributional Model of Meaning , 2014 .

[30]  J. Lambek.  The Mathematics of Sentence Structure , 1958 .

[31]  Elham Kashefi,et al.  A Quantum-Theoretic Approach to Distributional Semantics , 2013, NAACL.

[32]  Samson Abramsky,et al.  A categorical semantics of quantum protocols , 2004, Proceedings of the 19th Annual IEEE Symposium on Logic in Computer Science, 2004..

[33]  Dimitri Kartsaklis,et al.  Prior Disambiguation of Word Tensors for Constructing Sentence Vectors , 2013, EMNLP.

[34]  James Richard Curran,et al.  From distributional to semantic similarity , 2004 .

[35]  G. M. Kelly,et al.  Coherence for compact closed categories , 1980 .

[36]  Dominic Widdows,et al.  Orthogonal Negation in Vector Spaces for Modelling Word-Meanings and Document Retrieval , 2003, ACL.