The role of syntactic dependencies in compositional distributional semantics

Abstract This article provides a preliminary semantic framework for Dependency Grammar in which lexical words are semantically defined as contextual distributions (sets of contexts) while syntactic dependencies are compositional operations on word distributions. More precisely, any syntactic dependency uses the contextual distribution of the dependent word to restrict the distribution of the head, and makes use of the contextual distribution of the head to restrict that of the dependent word. The interpretation of composite expressions and sentences, which are analyzed as a tree of binary dependencies, is performed by restricting the contexts of words dependency by dependency in a left-to-right incremental way. Consequently, the meaning of the whole composite expression or sentence is not a single representation, but a list of contextualized senses, namely the restricted distributions of its constituent (lexical) words. We report the results of two large-scale corpus-based experiments on two different natural language processing applications: paraphrasing and compositional translation.

[1]  José Ramom Pichel Campos,et al.  Learning Spanish-Galician Translation Equivalents Using a Comparable Corpus and a Bilingual Dictionary , 2008, CICLing.

[2]  Stefan Thater,et al.  Contextualizing Semantic Representations Using Syntactically Enriched Vector Models , 2010, ACL.

[3]  Ann A. Copestake.  Lexicalised compositionality , 2011 .

[4]  Silvia Bernardini,et al.  The WaCky wide web: a collection of very large linguistically processed web-crawled corpora , 2009, Lang. Resour. Evaluation.

[5]  Gregory Grefenstette,et al.  Evaluation Techniques for Automatic Semantic Extraction: Comparing Syntactic and Window Based Approaches , 1996 .

[6]  Todd R. Ferretti,et al.  Thematic Roles as Verb-specific Concepts , 1997 .

[7]  Marco Baroni,et al.  Frege in Space: A Program for Composition Distributional Semantics , 2014, LILT.

[8]  Katrin Erk.  Towards a semantics for distributional representations , 2013, IWCS.

[9]  Peter D. Turney.  Domain and Function: A Dual-Space Model of Semantic Relations and Compositions , 2012, J. Artif. Intell. Res..

[10]  Mirella Lapata,et al.  Vector-based Models of Semantic Composition , 2008, ACL.

[11]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .

[12]  E. Guevara.  A Regression Model of Adjective-Noun Compositionality in Distributional Semantics , 2010 .

[13]  Lucien Tesnière.  Éléments de syntaxe structurale , 1959 .

[14]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[15]  Gregory Grefenstetti,et al.  Evaluation techniques for automatic semantic extraction: comparing syntactic and window based approaches , 1996 .

[16]  Elisabetta Jezek,et al.  What lexical sets tell us about conceptual categories , 2010 .

[17]  Mark Steedman,et al.  Surface structure and interpretation , 1996, Linguistic inquiry.

[18]  Mirella Lapata,et al.  Language Models Based on Semantic Composition , 2009, EMNLP.

[19]  Dov M. Gabbay,et al.  Language Understanding: A Procedural Perspective , 1996, LACL.

[20]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[21]  Pablo Gamallo Otero.  Learning bilingual lexicons from comparable English and Spanish corpora , 2007, MTSUMMIT.

[22]  Roberto Navigli,et al.  Word sense disambiguation: A survey , 2009, CSUR.

[23]  Reinhard Rapp,et al.  Automatic Identification of Word Translations from Unrelated English and German Corpora , 1999, ACL.

[24]  Matthias Schlesewsky,et al.  On incremental interpretation: Degrees of meaning accessed during sentence comprehension , 2004 .

[25]  Pablo Gamallo,et al.  Clustering Syntactic Positions with Similar Semantic Requirements , 2005, CL.

[26]  Steven K. Strange,et al.  On Aristotle's categories , 1992 .

[27]  Claire Lemaire,et al.  Extraction of Domain-Specific Bilingual Lexicon from Comparable Corpora: Compositional Translation and Ranking , 2012, COLING.

[28]  Katrin Erk,et al.  A Structured Vector Space Model for Word Meaning in Context , 2008, EMNLP.

[29]  Ludger Jansen,et al.  Aristotle’s Categories , 2007 .

[30]  D. Davidson.  The Individuation of Events , 1969 .

[31]  Timothy Baldwin,et al.  Noun-Noun Compound Machine Translation A Feasibility Study on Shallow Processing , 2003, Proceedings of the ACL 2003 workshop on Multiword expressions analysis, acquisition and treatment -.

[32]  James Pustejovsky,et al.  The Generative Lexicon , 1995, CL.

[33]  Pablo Gamallo Otero,et al.  The Meaning of Syntactic Dependencies , 2013 .

[34]  A. Meillet.  Linguistique historique et linguistique générale , 1908 .

[35]  Christiane Fellbaum,et al.  A Semantic Network of English: The Mother of All WordNets , 1998, Comput. Humanit..

[36]  Pablo Gamallo Otero,et al.  A grammatical formalism based on patterns of part of speech tags , 2011 .

[37]  Georgiana Dinu,et al.  DISSECT - DIStributional SEmantics Composition Toolkit , 2013, ACL.

[38]  Patrick Hanks,et al.  Lexical Analysis: Norms and Exploitations , 2013 .

[39]  Barbara H. Partee,et al.  Privative Adjectives: Subsective plus Coercion , 2003 .

[40]  Giovanni Soda,et al.  Wide Coverage Incremental Parsing by Learning Attachment Preferences , 2001, AI*IA.

[41]  Philip Koehn,et al.  Statistical Machine Translation , 2010, EAMT.

[42]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[43]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[44]  D. Sperber,et al.  Relevance: Communication and Cognition , 1989 .

[45]  Ioannis Korkontzelos,et al.  Estimating Linear Models for Compositional Distributional Semantics , 2010, COLING.

[46]  David Milward.  Dynamics, Dependency Grammar And Incremental Interpretation , 1992, COLING.

[47]  Pascale Fung,et al.  An IR Approach for Translating New Words from Nonparallel, Comparable Texts , 1998, ACL.

[48]  Pablo Gamallo Otero.  Cognitive characterisation of basic grammatical structures , 2003 .

[49]  Richard Hudson,et al.  The psychological reality of syntactic dependency relations , 2003 .

[50]  Dov M. Gabbay,et al.  Dynamic syntax - the flow of language understanding , 2000 .

[51]  Sylvain Kahane,et al.  The Meaning-Text Theory , 2003 .

[52]  Emmanuel Morin,et al.  Revising the Compositional Method for Terminology Acquisition from Comparable Corpora , 2012, COLING.

[53]  Stephen Clark,et al.  Mathematical Foundations for a Compositional Distributional Model of Meaning , 2010, ArXiv.

[54]  R. Langacker.  Foundations of cognitive grammar , 1983 .

[55]  Marco Baroni,et al.  Composition in Distributional Semantics , 2013, Lang. Linguistics Compass.

[56]  Michael K. Tanenhaus,et al.  Lexical structure and language comprehension , 1989 .

[57]  László Dezsö,et al.  Universal Grammar , 1981, Certainty in Action.

[58]  Georgiana Dinu,et al.  General estimation and evaluation of compositional distributional semantic models , 2013, CVSM@ACL.

[59]  Jeroen Groenendijk,et al.  Dynamic predicate logic , 1991 .

[60]  Stephen Clark,et al.  Concrete Sentence Spaces for Compositional Distributional Models of Meaning , 2010, IWCS.

[61]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[62]  Gregory Grefenstette,et al.  The World Wide Web as a Resource for Example-Based Machine Translation Tasks , 1999, TC.