A Type-Driven Vector Semantics for Ellipsis with Anaphora Using Lambek Calculus with Limited Contraction

We develop a vector space semantics for verb phrase ellipsis with anaphora using type-driven compositional distributional semantics based on the Lambek calculus with limited contraction (LCC) of Jäger (Anaphora and type logical grammar, Springer, Berlin, 2006). Distributional semantics has a lot to say about the statistical collocation based meanings of content words, but provides little guidance on how to treat function words. Formal semantics on the other hand, has powerful mechanisms for dealing with relative pronouns, coordinators, and the like. Type-driven compositional distributional semantics brings these two models together. We review previous compositional distributional models of relative pronouns, coordination and a restricted account of ellipsis in the DisCoCat framework of Coecke et al. (Mathematical foundations for a compositional distributional model of meaning, 2010. arXiv:1003.4394, Ann Pure Appl Log 164(11):1079–1100, 2013). We show how DisCoCat cannot deal with general forms of ellipsis, which rely on copying of information, and develop a novel way of connecting typelogical grammar to distributional semantics by assigning vector interpretable lambda terms to derivations of LCC in the style of Muskens and Sadrzadeh (in: Amblard, de Groote, Pogodalla, Retoré (eds) Logical aspects of computational linguistics, Springer, Berlin, 2016). What follows is an account of (verb phrase) ellipsis in which word meanings can be copied: the meaning of a sentence is now a program with non-linear access to individual word embeddings. We present the theoretical setting, work out examples, and demonstrate our results with a state of the art distributional model on an extended verb disambiguation dataset.

[1]  Mehrnoosh Sadrzadeh,et al.  Lambek vs. Lambek: Functorial vector space semantics and string diagrams for Lambek calculus , 2013, Ann. Pure Appl. Log..

[2]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[3]  Gerhard Jaeger,et al.  A Multi-Modal Analysis of Anaphora and Ellipsis , 1998 .

[4]  Gerhard Jäger.  Anaphora and Type Logical Grammar , 2005 .

[5]  Reinhard Muskens,et al.  Language, Lambdas, and Logic , 2003 .

[6]  Bart Jacobs Institute for Computing,et al.  Bases as Coalgebras , 2011, CALCO.

[7]  Michael Moortgat,et al.  Lexical and Derivational Meaning in Vector-Based Models of Relativisation , 2017, ArXiv.

[8]  Mehrnoosh Sadrzadeh.  Quantifier Scope in Categorical Compositional Distributional Semantics , 2016, SLPCS@QPL.

[9]  Mehrnoosh Sadrzadeh,et al.  Experimenting with transitive verbs in a DisCoCat , 2011, GEMS.

[10]  Joachim Lambek,et al.  Type Grammar Revisited , 1997, LACL.

[11]  Glyn Morrill,et al.  On Calculus of Displacement , 2010, TAG.

[12]  Gijs Jasper Wijnholds,et al.  A Proof-Theoretic Approach to Scope Ambiguity in Compositional Vector Space Models , 2018, J. Lang. Model..

[13]  Stephen Clark,et al.  A Type-Driven Tensor-Based Semantics for CCG , 2014, EACL 2014.

[14]  Dusko Pavlovic,et al.  A new description of orthogonal bases , 2008, Mathematical Structures in Computer Science.

[15]  B. Coecke.  Introducing categories to the practicing physicist , 2008, 0808.1032.

[16]  Glyn Morrill,et al.  Grammar logicised: relativisation , 2017 .

[17]  Stephen Clark,et al.  The Frobenius anatomy of word meanings I: subject and object relative pronouns , 2013, J. Log. Comput..

[18]  S. Abramsky.  No-Cloning In Categorical Quantum Mechanics , 2009, 0910.2401.

[19]  G. Wijnholds.  Categorical Foundations for Extended Compositional Distributional Models of Meaning , 2014 .

[20]  Mehrnoosh Sadrzadeh,et al.  Classical Copying versus Quantum Entanglement in Natural Language: The Case of VP-ellipsis , 2018, CAPNS@QI.

[21]  Michael Moortgat,et al.  Categorial Type Logics , 1997, Handbook of Logic and Language.

[22]  Dimitri Kartsaklis,et al.  Verb Phrase Ellipsis using Frobenius Algebras in Categorical Compositional Distributional Semantics , 2016 .

[23]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[24]  Stephen Clark,et al.  Mathematical Foundations for a Compositional Distributional Model of Meaning , 2010, ArXiv.

[25]  Mehrnoosh Sadrzadeh,et al.  Static and Dynamic Vector Semantics for Lambda Calculus Models of Natural Language , 2018, J. Lang. Model..

[26]  Alessandro Lenci,et al.  Distributional semantics in linguistic and cognitive research , 2008 .

[27]  Dimitri Kartsaklis,et al.  Evaluating Neural Word Representations in Tensor-Based Compositional Settings , 2014, EMNLP.

[28]  Raffaella Bernardi,et al.  There Is No Logical Negation Here, But There Are Alternatives: Modeling Conversational Negation with Distributional Semantics , 2016, Computational Linguistics.

[29]  Mirella Lapata,et al.  Vector-based Models of Semantic Composition , 2008, ACL.

[30]  Stephen Clark,et al.  The Frobenius anatomy of word meanings II: possessive relative pronouns , 2014, J. Log. Comput..

[31]  Mehrnoosh Sadrzadeh,et al.  Multi-Step Regression Learning for Compositional Distributional Semantics , 2013, IWCS.

[32]  Yusuke Kubota,et al.  Pseudogapping as Pseudo-VP-Ellipsis , 2014, Linguistic Inquiry.

[33]  Richard Montague,et al.  ENGLISH AS A FORMAL LANGUAGE , 1975 .

[34]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[35]  Dimitri Kartsaklis,et al.  Prior Disambiguation of Word Tensors for Constructing Sentence Vectors , 2013, EMNLP.

[36]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[37]  Samson Abramsky,et al.  Computational Interpretations of Linear Logic , 1993, Theor. Comput. Sci..

[38]  Generalising DISCONTINUITYGlyn,et al.  Generalising Discontinuity , 1996 .

[39]  Mehrnoosh Sadrzadeh,et al.  Experimental Support for a Categorical Compositional Distributional Model of Meaning , 2011, EMNLP.

[40]  Petra Hendriks,et al.  Comparatives and Categorial Grammar , 1995 .

[41]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[42]  Stuart M. Shieber,et al.  Ellipsis and higher-order unification , 1991 .

[43]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[44]  Philippe de Groote,et al.  Towards Abstract Categorial Grammars , 2001, ACL.

[45]  Dimitri Kartsaklis.  Coordination in Categorical Compositional Distributional Semantics , 2016, SLPCS@QPL.

[46]  Mehrnoosh Sadrzadeh,et al.  A generalised quantifier theory of natural language in categorical compositional distributional semantics with bialgebras , 2016, Mathematical Structures in Computer Science.

[47]  Mehrnoosh Sadrzadeh,et al.  Context Update for Lambdas and Vectors , 2016, LACL.