What Company Do Semantically Ambiguous Words Keep? Insights from Distributional Word Vectors

The diversity of a word’s contexts affects its acquisition and processing. Can differences between word types such as monosemes (unambiguous words), polysemes (multiple related senses), and homonyms (multiple unrelated meanings) be related to distributional properties of these words? We tested for traces of number and relatedness of meaning in vector representations by comparing the distance between words of each type and vector representations of various “contexts”: their dictionary definitions (an extreme disambiguating context), their use in film subtitles (a natural context), and their semantic neighbours in vector space (a vector-space-internal context). Whereas dictionary definitions reveal a three-way split between our word types, the other two contexts produced a two-way split between ambiguous and unambiguous words. These inconsistencies align with some discrepancies in behavioural studies and present a paradox regarding how models learn meaning relatedness despite natural contexts seemingly lacking such relatedness. We argue that viewing ambiguity as a continuum could resolve many of these issues.

[1]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[2]  Georgiana Dinu,et al.  Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors , 2014, ACL.

[3]  Sanjeev Arora,et al.  Linear Algebraic Structure of Word Senses, with Applications to Polysemy , 2016, TACL.

[4]  D. Titone,et al.  Making sense of word senses: the comprehension of polysemy depends on sense overlap. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[5]  Blair C Armstrong,et al.  SOS! An algorithm and software for the stochastic optimization of stimuli , 2012, Behavior research methods.

[6]  Blair C Armstrong,et al.  eDom: Norming software and relative meaning frequencies for 544 English homonyms , 2012, Behavior Research Methods.

[7]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[8]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[9]  D. Balota,et al.  Moving beyond Coltheart’s N: A new measure of orthographic similarity , 2008, Psychonomic bulletin & review.

[10]  Melody Dye,et al.  Context as an Organizing Principle of the Lexicon , 2017 .

[11]  Curt Burgess,et al.  Representing and resolving semantic ambiguity: A contribution from high-dimensional memory modeling. , 2001 .

[12]  Fritz Günther,et al.  LSAfun - An R package for computations based on Latent Semantic Analysis , 2014, Behavior Research Methods.

[13]  Katrin Erk,et al.  Vector Space Models of Word Meaning and Phrase Meaning: A Survey , 2012, Lang. Linguistics Compass.

[14]  G. Murphy,et al.  The Representation of Polysemous Words , 2001 .

[15]  Marc Brysbaert,et al.  Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English , 2009, Behavior research methods.

[16]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[17]  Ewan Klein,et al.  Natural Language Processing with Python , 2009 .

[18]  Linda B. Smith,et al.  Infants rapidly learn word-referent mappings via cross-situational statistics , 2008, Cognition.

[19]  Murhaf Fares,et al.  Word vectors, reuse, and replicability: Towards a community repository of large-text resources , 2017, NODALIDA.

[20]  S. Lupker,et al.  Ambiguity and relatedness effects in semantic tasks: Are they due to semantic coding? , 2006 .

[21]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[22]  David C. Plaut,et al.  Disparate semantic ambiguity effects from semantic processing dynamics rather than qualitative task differences , 2016 .

[23]  Brent Kievit-Kylar,et al.  Organizing the space and behavior of semantic models , 2014, CogSci.

[24]  Chen Yu,et al.  A Bootstrapping Model of Frequency and Context Effects in Word Learning , 2017, Cogn. Sci..

[25]  Danielle S. McNamara,et al.  Computational Methods to Extract Meaning From Text and Advance Theories of Human Cognition , 2011, Top. Cogn. Sci..

[26]  W. Marslen-Wilson,et al.  Making Sense of Semantic Ambiguity: Semantic Competition in Lexical Access , 2002 .