Categorical Vector Space Semantics for Lambek Calculus with a Relevant Modality

We develop a categorical compositional distributional semantics for Lambek Calculus with a Relevant Modality !L*, which has a limited edition of the contraction and permutation rules. The categorical part of the semantics is a monoidal biclosed category with a coalgebra modality, very similar to the structure of a Differential Category. We instantiate this category to finite dimensional vector spaces and linear maps via "quantisation" functors and work with three concrete interpretations of the coalgebra modality. We apply the model to construct categorical and concrete semantic interpretations for the motivating example of !L*: the derivation of a phrase with a parasitic gap. The effectiveness of the concrete interpretations are evaluated via a disambiguation task, on an extension of a sentence disambiguation dataset to parasitic gap phrases, using BERT, Word2Vec, and FastText vectors and Relational tensors.

[1]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[2]  Martin Hyland,et al.  Glueing and orthogonality for models of linear logic , 2003, Theor. Comput. Sci..

[3]  Max I. Kanovich,et al.  Undecidability of the Lambek Calculus with a Relevant Modality , 2015, FG.

[4]  Prakash Panangaden,et al.  Fock Space: A Model of Linear Exponential Types , 1994 .

[5]  John C. Baez,et al.  Physics, Topology, Logic and Computation: A Rosetta Stone , 2009, 0903.0340.

[6]  Mehrnoosh Sadrzadeh,et al.  Lambek vs. Lambek: Functorial vector space semantics and string diagrams for Lambek calculus , 2013, Ann. Pure Appl. Log..

[7]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[8]  Mehrnoosh Sadrzadeh,et al.  Classical Copying versus Quantum Entanglement in Natural Language: The Case of VP-ellipsis , 2018, CAPNS@QI.

[9]  Stephen Clark,et al.  The Frobenius anatomy of word meanings I: subject and object relative pronouns , 2013, J. Log. Comput..

[10]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[11]  Alain Bruguieres,et al.  Hopf monads , 2006 .

[12]  Mehrnoosh Sadrzadeh,et al.  A Type-Driven Vector Semantics for Ellipsis with Anaphora Using Lambek Calculus with Limited Contraction , 2019, Journal of Logic, Language and Information.

[13]  Gregory Grefenstette,et al.  Explorations in automatic thesaurus discovery , 1994 .

[14]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[15]  J. Robin B. Cockett,et al.  Differential categories , 2006, Mathematical Structures in Computer Science.

[16]  M. Pickering,et al.  Processing ambiguous verbs: evidence from eye movements. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[17]  Dimitri Kartsaklis,et al.  Prior Disambiguation of Word Tensors for Constructing Sentence Vectors , 2013, EMNLP.

[18]  Gijs Jasper Wijnholds,et al.  Coherent Diagrammatic Reasoning in Compositional Distributional Semantics , 2017, WoLLIC.

[19]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[20]  Dimitri Kartsaklis,et al.  Evaluating Neural Word Representations in Tensor-Based Compositional Settings , 2014, EMNLP.

[21]  Andre Scedrov,et al.  Bounded Linear Logic: A Modular Approach to Polynomial-Time Computability , 1992, Theor. Comput. Sci..

[22]  Donald Yau,et al.  Categories , 2021, 2-Dimensional Categories.

[23]  Dimitri Kartsaklis,et al.  Sentence entailment in compositional distributional semantics , 2015, Annals of Mathematics and Artificial Intelligence.

[24]  G. Salton,et al.  A document retrieval system for man-machine interaction , 1964, ACM National Conference.

[25]  Johan van Benthem,et al.  The Lambek Calculus , 1988 .

[26]  Yves Lafont,et al.  Soft linear logic and polynomial time , 2004, Theor. Comput. Sci..

[27]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[28]  Michael Moortgat,et al.  Lexical and Derivational Meaning in Vector-Based Models of Relativisation , 2017, ArXiv.

[29]  Glyn Morrill,et al.  Proof Figures and Structural Operators for Categorial Grammar , 1991, EACL.

[30]  James Richard Curran,et al.  From distributional to semantic similarity , 2004 .

[31]  J. Humphreys.  Introduction to Lie Algebras and Representation Theory , 1973 .

[32]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[33]  Dimitri Kartsaklis,et al.  Separating Disambiguation from Composition in Distributional Semantics , 2013, CoNLL.

[34]  Michael Moortgat.  Multimodal Linguistic Inference , 1995, Log. J. IGPL.

[35]  Olga Vechtomova,et al.  Book Review: Introduction to Information Retrieval by Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze , 2009, CL.

[36]  知秀 柴田.  5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding , 2020 .

[37]  Mehrnoosh Sadrzadeh,et al.  A Frobenius Algebraic Analysis for Parasitic Gaps , 2020, FLAP.

[38]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[39]  Glyn Morrill,et al.  Grammar logicised: relativisation , 2017 .

[40]  P. Selinger.  A Survey of Graphical Languages for Monoidal Categories , 2009, 0908.3347.

[41]  Glyn Morrill,et al.  A Note on movement in logical grammar , 2019, J. Lang. Model..

[42]  Glyn Morrill,et al.  On the Logic of Expansion in Natural Language , 2016, LACL.

[43]  Mehrnoosh Sadrzadeh,et al.  Experimental Support for a Categorical Compositional Distributional Model of Meaning , 2011, EMNLP.

[44]  John B. Goodenough,et al.  Contextual correlates of synonymy , 1965, CACM.

[45]  Mehrnoosh Sadrzadeh,et al.  Evaluating Composition Models for Verb Phrase Elliptical Sentence Embeddings , 2019, NAACL.

[46]  Mehrnoosh Sadrzadeh,et al.  Concrete Models and Empirical Evaluations for the Categorical Compositional Distributional Model of Meaning , 2015, CL.