Affixation in semantic space: Modeling morpheme meanings with compositional distributional semantics.

The present work proposes a computational model of morpheme combination at the meaning level. The model moves from the tenets of distributional semantics, and assumes that word meanings can be effectively represented by vectors recording their co-occurrence with other words in a large text corpus. Given this assumption, affixes are modeled as functions (matrices) mapping stems onto derived forms. Derived-form meanings can be thought of as the result of a combinatorial procedure that transforms the stem vector on the basis of the affix matrix (e.g., the meaning of nameless is obtained by multiplying the vector of name with the matrix of -less). We show that this architecture accounts for the remarkable human capacity of generating new words that denote novel meanings, correctly predicting semantic intuitions about novel derived forms. Moreover, the proposed compositional approach, once paired with a whole-word route, provides a new interpretative framework for semantic transparency, which is here partially explained in terms of ease of the combinatorial procedure and strength of the transformation brought about by the affix. Model-based predictions are in line with the modulation of semantic transparency on explicit intuitions about existing words, response times in lexical decision, and morphological priming. In conclusion, we introduce a computational model to account for morpheme combination at the meaning level. The model is data-driven, theoretically sound, and empirically supported, and it makes predictions that open new research avenues in the domain of semantic processing. (PsycINFO Database Record

[1]  H. Marchand Categories And Types Of Present Day English Word Formation , 1971 .

[2]  F. E. Satterthwaite An approximate distribution of estimates of variance components. , 1946, Biometrics.

[3]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[4]  J. Firth,et al.  Papers in linguistics, 1934-1951 , 1957 .

[5]  J. Berko The Child's Learning of English Morphology , 1958 .

[6]  H. Marchand The categories and types of present-day English word-formation : a synchronic-diachronic approach , 1960 .

[7]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[8]  K. Forster,et al.  Lexical storage and retrieval of prefixed words , 1975 .

[9]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[10]  S. Pinker,et al.  On language and connectionism: Analysis of a parallel distributed processing model of language acquisition , 1988, Cognition.

[11]  William E. Nagy,et al.  The acquisition of English derivational morphology , 1989 .

[12]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[13]  G. Miller,et al.  Contextual correlates of semantic similarity , 1991 .

[14]  Geoffrey E. Hinton Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[15]  Dominiek Sandra,et al.  The morphology of the mental lexicon: Internal word structure viewed from a psycholinguistic perspective , 1994 .

[16]  T. A. Cartwright,et al.  Distributional regularity and phonotactic constraints are useful for segmentation , 1996, Cognition.

[17]  Robert Schreuder,et al.  Modelling the processing of morphologically complex words. , 1996 .

[18]  R. H. Baayen,et al.  The CELEX Lexical Database (CD-ROM) , 1996 .

[19]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[20]  Robert Schreuder,et al.  How Complex Simplex Words can be , 1997 .

[21]  K. Forster,et al.  What can we learn from the morphology of Hebrew? A masked-priming investigation of morphological representation. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[22]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[23]  Gary Libben,et al.  Semantic Transparency in the Processing of Compounds: Consequences for Representation, Processing, and Impairment , 1998, Brain and Language.

[24]  L. Feldman,et al.  Morphological Priming: The Role of Prime Duration, Semantic Transparency, and Affix Position , 1999, Brain and Language.

[25]  Matti Laine Meaning Analysis of Inflected Words , 1999 .

[26]  H. Clahsen,et al.  Lexical entries and rules of language: A multidisciplinary study of German inflection , 1999, Behavioral and Brain Sciences.

[27]  Matthew H. Davis,et al.  Morphological and semantic effects in visual word recognition: A time-course study , 2000 .

[28]  L. Feldman,et al.  Are morphological effects distinguishable from the effects of shared meaning and shared form? , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[29]  Robert Schreuder,et al.  Effects of Family Size for Complex Words , 2000 .

[30]  Mark T. Keane,et al.  Efficient creativity: constraint-guided conceptual combination , 2000, Cogn. Sci..

[31]  David C. Plaut,et al.  Are non-semantic morphological effects incompatible with a distributed connectionist approach to lexical processing? , 2000 .

[32]  Nivja H. de Jong,et al.  The morphological family size effect and morphology , 2000 .

[33]  R. Harald Baayen,et al.  Parsing and productivity , 2002 .

[34]  Bruce Hayes,et al.  Modeling English Past Tense Intuitions with Minimal Generalization , 2002, SIGMORPHON.

[35]  R. Jackendoff Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002 .

[36]  G. Murphy,et al.  The Big Book of Concepts , 2002 .

[37]  S. Pinker,et al.  The past and future of the past tense , 2002, Trends in Cognitive Sciences.

[38]  James L. McClelland,et al.  Rules or connections in past-tense inflections: what does the evidence rule out? , 2002, Trends in Cognitive Sciences.

[39]  Matthew H. Davis,et al.  Reading morphologically complex words: Some thoughts from masked priming. , 2003 .

[40]  J. Segui,et al.  Morphological priming without morphological relationship , 2003 .

[41]  C. Habel,et al.  Language , 1931, NeuroImage.

[42]  M. Taft Morphological Decomposition and the Reverse Base Frequency Effect , 2004, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[43]  Sophia Ananiadou,et al.  On the definition of word , 2004, Machine Translation.

[44]  Scott McDonald,et al.  A Distributional Model of Semantic Context Effects in Lexical Processing , 2004, ACL.

[45]  E. Wagenmakers,et al.  AIC model selection using Akaike weights , 2004, Psychonomic bulletin & review.

[46]  R. Baayen,et al.  Morphological family size in a morphologically rich language: the case of Finnish compared with Dutch and Hebrew. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[47]  A. Goldberg,et al.  The English Resultative as a Family of Constructions , 2004 .

[48]  Rochelle Lieber,et al.  Morphology and Lexical Semantics , 2004 .

[49]  R. Holloway The broth in my brother ’ s brothel : Morpho-orthographic segmentation in visual word recognition , 2005 .

[50]  J. Grainger,et al.  Masked cross-modal morphological priming: Unravelling morpho-orthographic and morpho-semantic influences in early word recognition , 2005 .

[51]  F. Meunier,et al.  Morphological decomposition in early visual word processing , 2005 .

[52]  R. Baayen,et al.  Shifting paradigms: gradient structure in morphology , 2005, Trends in Cognitive Sciences.

[53]  Jeffrey L. Stuart Linear Algebra, 3rd ed./Linear Algebra and its Applications, 3rd ed./Linear Algebra: A Geometric Approach/Introduction to Linear Algebra, 3rd ed , 2005 .

[54]  R. Baayen,et al.  Morphological influences on the recognition of monosyllabic monomorphemic words , 2006 .

[55]  Alec Marantz,et al.  A single route, full decomposition model of morphological complexity: MEG evidence , 2006 .

[56]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[57]  M. Tomalin Linguistics and the formal sciences , 2006 .

[58]  L. Feldman,et al.  Semantic influences on morphological facilitation: Concreteness and family size , 2006 .

[59]  F. Meunier,et al.  Morphological decomposition and semantic integration in word processing , 2007 .

[60]  R. Frost,et al.  SOA does not Reveal the Absolute Time Course of Cognitive Processing in Fast Priming Experiments. , 2007, Journal of memory and language.

[61]  Mark S. Seidenberg,et al.  Graded semantic and phonological similarity effects in priming: evidence for a distributed connectionist approach to morphology. , 2007, Journal of experimental psychology. General.

[62]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[63]  Mirella Lapata,et al.  Dependency-Based Construction of Semantic Space Models , 2007, CL.

[64]  Lee H. Wurm,et al.  Lexical dynamics for low-frequency complex words: A regression study across tasks and modalities , 2007 .

[65]  Rebecca Treiman,et al.  The English Lexicon Project , 2007, Behavior research methods.

[66]  Brian T. Gold,et al.  Neural Correlates of Morphological Decomposition during Visual Word Recognition , 2007, Journal of Cognitive Neuroscience.

[67]  J. Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: A computational study , 2007, Behavior research methods.

[68]  Magnus Sahlgren,et al.  The Distributional Hypothesis , 2008 .

[69]  Matthew H. Davis,et al.  Morphological decomposition based on the analysis of orthography , 2008 .

[70]  Dennis Norris,et al.  Perception as evidence accumulation and Bayesian inference: insights from masked priming. , 2008, Journal of experimental psychology. General.

[71]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[72]  Alexander Pollatsek,et al.  The role of semantic transparency in the processing of English compound words. , 2008, British journal of psychology.

[73]  D. Balota,et al.  Moving beyond Coltheart’s N: A new measure of orthographic similarity , 2008, Psychonomic bulletin & review.

[74]  Jay G Rueckl,et al.  Are CORNER and BROTHER morphologically complex? Not in the long term , 2008, Language and cognitive processes.

[75]  R. Baayen,et al.  Paradigms bit by bit : an information-theoretic approach to the processing of paradigmatic structure in inflection and derivation , 2008 .

[76]  William D. Marslen-Wilson,et al.  Early decomposition in visual word recognition: Dissociating morphology, form, and meaning , 2008, Language and cognitive processes.

[77]  Alexander Pollatsek,et al.  The role of semantic transparency in the processing of Finnish compound words , 2005 .

[78]  Christina L. Gagné,et al.  Constituent integration during the processing of compound words: Does it involve the use of relational structures? , 2009 .

[79]  Patrick A. O’Connor,et al.  Early morphological processing is morphosemantic and not simply morpho-orthographic: A violation of form-then-meaning accounts of word recognition , 2009, Psychonomic bulletin & review.

[80]  Kevin Diependaele,et al.  Semantic transparency and masked morphological priming: The case of prefixed words , 2009, Memory & cognition.

[81]  Stephen Clark,et al.  Mathematical Foundations for a Compositional Distributional Model of Meaning , 2010, ArXiv.

[82]  Alexandros Nanopoulos,et al.  On the existence of obstinate results in vector space models , 2010, SIGIR.

[83]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[84]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[85]  Mirella Lapata,et al.  Measuring Distributional Similarity in Context , 2010, EMNLP.

[86]  Massimo Poesio,et al.  Strudel: A distributional semantic model based on properties and types , 2010 .

[87]  Katrin Erk,et al.  A Flexible, Corpus-Driven Model of Regular and Inverse Selectional Preferences , 2010, CL.

[88]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[89]  John Goldsmith,et al.  Segmentation and morphology , 2010 .

[90]  Ioannis Korkontzelos,et al.  Estimating Linear Models for Compositional Distributional Semantics , 2010, COLING.

[91]  E. Guevara A Regression Model of Adjective-Noun Compositionality in Distributional Semantics , 2010 .

[92]  Geert Booij,et al.  Construction Morphology , 2010, Lang. Linguistics Compass.

[93]  Victor Kuperman,et al.  Using Amazon Mechanical Turk for linguistic research , 2010 .

[94]  M. Coltheart,et al.  ‘Fell’ primes ‘fall’, but does ‘bell’ prime ‘ball’? Masked priming with irregularly-inflected primes , 2010 .

[95]  Emmanuel Keuleers,et al.  Fast morphological effects in first and second language word recognition , 2011 .

[96]  Pirita Pyykkönen,et al.  Sub- and Supralexical Information in Early Phases of Lexical Access , 2011, Front. Psychology.

[97]  Christina L. Gagné,et al.  Benefits and costs of lexical decomposition and semantic integration during the processing of transparent and opaque English compounds , 2011 .

[98]  Dušica Filipović Đurđević,et al.  An amorphous model for morphological processing in visual comprehension based on naive discriminative learning. , 2011, Psychological review.

[99]  Mehrnoosh Sadrzadeh,et al.  Experimental Support for a Categorical Compositional Distributional Model of Meaning , 2011, EMNLP.

[100]  Jonathan Grainger,et al.  A Dual-Route Approach to Orthographic Processing , 2011, Front. Psychology.

[101]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[102]  Nina Kazanina,et al.  Decomposition of prefixed words in Russian. , 2011, Journal of experimental psychology. Learning, memory, and cognition.

[103]  Eva Maria Vecchi,et al.  (Linear) Maps of the Impossible: Capturing Semantic Anomalies in Distributional Space , 2011 .

[104]  Katrin Erk,et al.  Vector Space Models of Word Meaning and Phrase Meaning: A Survey , 2012, Lang. Linguistics Compass.

[105]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[106]  Sanjeev Arora,et al.  Learning Topic Models -- Going beyond SVD , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[107]  Laurie Beth Feldman,et al.  Semantic similarity influences early morphological priming in Serbian: A challenge to form-then-meaning accounts of word recognition , 2012, Psychonomic Bulletin & Review.

[108]  John A Bullinaria,et al.  Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD , 2012, Behavior Research Methods.

[109]  J. Bowers,et al.  Bayesian just-so stories in psychology and neuroscience. , 2012, Psychological bulletin.

[110]  Gemma Boleda,et al.  Distributional Semantics in Technicolor , 2012, ACL.

[111]  Nicolas J. Bourguignon,et al.  The temporal dynamics of inflected word recognition: A masked ERP priming study of French verbs , 2012, Neuropsychologia.

[112]  Marco Marelli,et al.  Frequency Effects in the Processing of Italian Nominal Compounds: Modulation of Headedness and Semantic Transparency , 2012 .

[113]  Yiu-Kei Tsang,et al.  Early Morphological Processing Is Sensitive to Morphemic Meanings: Evidence from Processing Ambiguous Morphemes. , 2013 .

[114]  Ted Briscoe,et al.  Capturing Anomalies in the Choice of Content Words in Compositional Distributional Semantic Space , 2013, RANLP.

[115]  Georgiana Dinu,et al.  General estimation and evaluation of compositional distributional semantic models , 2013, CVSM@ACL.

[116]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[117]  Frank Keller,et al.  Probabilistic Modeling of Discourse-Aware Sentence Processing , 2013, Top. Cogn. Sci..

[118]  Christopher D. Manning,et al.  Better Word Representations with Recursive Neural Networks for Morphology , 2013, CoNLL.

[119]  Georgiana Dinu,et al.  DISSECT - DIStributional SEmantics Composition Toolkit , 2013, ACL.

[120]  Louise McNally,et al.  Intensionality was only alleged: On adjective-noun composition in distributional semantics , 2013, IWCS.

[121]  Marco Marelli,et al.  Meaning is in the beholder’s eye: Morpho-semantic effects in masked priming , 2013, Psychonomic bulletin & review.

[122]  Victor Kuperman,et al.  Accentuate the Positive: Semantic Access in English Compounds , 2013, Front. Psychol..

[123]  Marco Marelli,et al.  Compositional-ly Derived Representations of Morphologically Complex Words in Distributional Semantics , 2013, ACL.

[124]  Angeliki Lazaridou,et al.  Fish Transporters and Miracle Homes: How Compositional Distributional Semantics can Help NP Parsing , 2013, EMNLP.

[125]  L. Feldman Modeling Morphological Processing , 2013 .

[126]  Stephen Clark Type-Driven Syntax and Semantics for Composing Meaning Vectors , 2013, Quantum Physics and Linguistics.

[127]  Hsuan-Chich Chen,et al.  Activation of morphemic meanings in processing opaque words , 2014, Psychonomic bulletin & review.

[128]  Hsueh-Cheng Wang,et al.  Predicting raters’ transparency judgments of English and Chinese morphological constituents using latent semantic analysis , 2013, Behavior Research Methods.

[129]  M. Marelli,et al.  Influence of verb and noun bases on reading aloud derived nouns: evidence from children with good and poor reading skills , 2014 .

[130]  P. Brockhoff,et al.  lmerTest: Tests for random and fixed effects for linear mixed effect models (lmer objects of lme4 package) , 2014 .

[131]  Marco Baroni,et al.  Frege in Space: A Program of Compositional Distributional Semantics , 2014 .

[132]  M. Laine,et al.  Testing the Stem Dominance Hypothesis: Meaning Analysis of Inflected Words and Prepositional Phrases , 2014, PloS one.

[133]  Stephen Clark,et al.  Vector Space Models of Lexical Meaning , 2015 .