Predictability of Distributional Semantics in Derivational Word Formation

Compositional distributional semantic models (CDSMs) have successfully been applied to the task of predicting the meaning of a range of linguistic constructions. Their performance on semi-compositional word formation process of (morphological) derivation, however, has been extremely variable, with no large-scale empirical investigation to date. This paper fills that gap, performing an analysis of CDSM predictions on a large dataset (over 30,000 German derivationally related word pairs). We use linear regression models to analyze CDSM performance and obtain insights into the linguistic factors that influence how predictable the distributional context of a derived word is going to be. We identify various such factors, notably part of speech, argument structure, and semantic regularity.

[1]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[2]  Marco Marelli,et al.  Compositional-ly Derived Representations of Morphologically Complex Words in Distributional Semantics , 2013, ACL.

[3]  Herbert H. Clark,et al.  When Nouns Surface as Verbs , 1979 .

[4]  Ido Dagan,et al.  Learning Entailment Rules for Unary Templates , 2008, COLING.

[5]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[6]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.

[7]  Helmut Schmidt,et al.  Probabilistic part-of-speech tagging using decision trees , 1994 .

[8]  Marco Baroni,et al.  Frege in Space: A Program for Composition Distributional Semantics , 2014, LILT.

[9]  Christopher D. Manning,et al.  Better Word Representations with Recursive Neural Networks for Morphology , 2013, CoNLL.

[10]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[11]  W. Dressler.  Word-Formation in Natural Morphology , 2005 .

[12]  U. Grömping.  Estimators of Relative Importance in Linear Regression Based on Variance Decomposition , 2007 .

[13]  P. Sen,et al.  Introduction to bivariate and multivariate analysis , 1981 .

[14]  益子 真由美.  Argument Structure , 1993, The Lexicon.

[15]  Eva Maria Vecchi,et al.  (Linear) Maps of the Impossible: Capturing Semantic Anomalies in Distributional Space , 2011 .

[16]  Bernd Bohnet,et al.  Very high accuracy and fast dependency parsing is not a contradiction , 2010, COLING 2010.

[17]  Katrin Erk,et al.  A Structured Vector Space Model for Word Meaning in Context , 2008, EMNLP.

[18]  Jan Snajder,et al.  Derivational Smoothing for Syntactic Distributional Semantics , 2013, ACL.

[19]  Marco Marelli,et al.  Semantic Transparency in Free Stems: The Effect of Orthography-Semantics Consistency on Word Recognition , 2015, Quarterly journal of experimental psychology.

[20]  J. Hay.  Causes and Consequences of Word Structure , 2003 .

[21]  Martin Haspelmath,et al.  Language typology and language universals : an international handbook , 2001 .

[22]  Prefixes in English Word Formation , 1995 .

[23]  Frans Plank,et al.  Morphologische (Ir-)Regularitäten : Aspekte der Wortstrukturtheorie , 1981 .

[24]  Phil Blunsom,et al.  Multilingual Models for Compositional Distributed Semantics , 2014, ACL.

[25]  Jan Snajder,et al.  Obtaining a Better Understanding of Distributional Models of German Derivational Morphology , 2015, IWCS.

[26]  Gertrud Faaß,et al.  SdeWaC - A Corpus of Parsable Sentences from the Web , 2013, GSCL.

[27]  Jan Snajder,et al.  DErivBase: Inducing and Evaluating a Derivational Morphology Resource for German , 2013, ACL.

[28]  Georgiana Dinu,et al.  DISSECT - DIStributional SEmantics Composition Toolkit , 2013, ACL.

[29]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[30]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[31]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[32]  Nizar Habash,et al.  A Categorial Variation Database for English , 2003, NAACL.

[33]  Antje Rossdeutscher,et al.  German particle verbs with 'auf': reconstructing their composition in a DRT-based framework , 2009 .

[34]  Philip Resnik,et al.  Inducing Frame Semantic Verb Classes from WordNet and LDOCE , 2004, ACL.

[35]  H. H. Clark.  The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. , 1973 .

[36]  Mihai Surdeanu,et al.  Ensemble Models for Dependency Parsing: Cheap and Good? , 2010, HLT-NAACL.

[37]  Stephen Clark,et al.  Mathematical Foundations for a Compositional Distributional Model of Meaning , 2010, ArXiv.

[38]  Ingo Plag,et al.  Word-Formation in English , 2018 .