Separating Disambiguation from Composition in Distributional Semantics

Most compositional-distributional models of meaning are based on ambiguous vector representations, where all the senses of a word are fused into the same vector. This paper provides evidence that the addition of a vector disambiguation step prior to the actual composition would be beneficial to the whole process, producing better composite representations. Furthermore, we relate this issue with the current evaluation practice, showing that disambiguation-based tasks cannot reliably assess the quality of composition. Using a word sense disambiguation scheme based on the generic procedure of Schutze (1998), we first provide a proof of concept for the necessity of separating disambiguation from composition. Then we demonstrate the benefits of an “unambiguous” system on a composition-only task.

[1]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[2]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[3]  Mirella Lapata,et al.  A Comparison of Vector-based Representations for Semantic Composition , 2012, EMNLP.

[4]  Ricardo J. G. B. Campello,et al.  On the Comparison of Relative Clustering Validity Criteria , 2009, SDM.

[5]  Walter Kintsch,et al.  Predication , 2001, Cogn. Sci..

[6]  E. Guevara.  A Regression Model of Adjective-Noun Compositionality in Distributional Semantics , 2010 .

[7]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[8]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[9]  Suresh Manandhar,et al.  Dynamic and Static Prototype Vectors for Semantic Composition , 2011, IJCNLP.

[10]  Xuchen Yao,et al.  Nonparametric Bayesian Word Sense Induction , 2011, Graph-based Methods for Natural Language Processing.

[11]  Mehrnoosh Sadrzadeh,et al.  Experimental Support for a Categorical Compositional Distributional Model of Meaning , 2011, EMNLP.

[12]  Suresh Manandhar,et al.  SemEval-2010 Task 14: Word Sense Induction &Disambiguation , 2010, SemEval@ACL.

[13]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[14]  M. Pickering,et al.  Processing ambiguous verbs: evidence from eye movements. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[15]  Mirella Lapata,et al.  Vector-based Models of Semantic Composition , 2008, ACL.

[16]  Dimitri Kartsaklis,et al.  A Unified Sentence Space for Categorical Distributional-Compositional Semantics: Theory and Experiments , 2012, COLING.

[17]  Bartosz Broda,et al.  Evaluation of clustering algorithms for word sense disambiguation , 2012, Int. J. Data Anal. Tech. Strateg..

[18]  Mehrnoosh Sadrzadeh,et al.  Experimenting with transitive verbs in a DisCoCat , 2011, GEMS.

[19]  André Hardy,et al.  An examination of procedures for determining the number of clusters in a data set , 1994 .

[20]  Mehrnoosh Sadrzadeh,et al.  Multi-Step Regression Learning for Compositional Distributional Semantics , 2013, IWCS.

[21]  Julia Hirschberg,et al.  V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure , 2007, EMNLP.

[22]  Christopher G. Chute,et al.  Cluster Stopping Rules for Word Sense Discrimination , 2006 .

[23]  Mehrnoosh Sadrzadeh,et al.  Quantum Physics and Linguistics - A Compositional, Diagrammatic Discourse , 2013, Quantum Physics and Linguistics.

[24]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[25]  Daniel Müllner,et al.  fastcluster: Fast Hierarchical, Agglomerative Clustering Routines for R and Python , 2013 .

[26]  Katrin Erk,et al.  A Structured Vector Space Model for Word Meaning in Context , 2008, EMNLP.

[27]  Daniel Müllner.  Fast Hierarchical Clustering Routines for R and Python , 2015 .

[28]  Christopher D. Manning,et al.  Learning Continuous Phrase Representations and Syntactic Parsing with Recursive Neural Networks , 2010 .