Obtaining a Better Understanding of Distributional Models of German Derivational Morphology

Predicting the (distributional) meaning of derivationally related words (read / read+er) from one another has recently been recognized as an instance of distributional compositional meaning construction. However, the properties of this task are not yet well understood. In this paper, we present an analysis of two such composition models on a set of German derivation patterns (e.g., -in, durch-). We begin by introducing a rank-based evaluation metric, which reveals the task to be challenging due to specific properties of German (compounding, capitalization). We also find that performance varies greatly between patterns and even among base-derived term pairs of the same pattern. A regression analysis shows that semantic coherence of the base and derived terms within a pattern, as well as coherence of the semantic shifts from base to derived terms, all significantly impact prediction quality.

[1]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[2]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[3]  Phil Blunsom,et al.  Compositional Morphology for Word Representations and Language Modelling , 2014, ICML.

[4]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.

[5]  Christopher D. Manning,et al.  Better Word Representations with Recursive Neural Networks for Morphology , 2013, CoNLL.

[6]  Bernd Bohnet,et al.  Top Accuracy and Fast Dependency Parsing is not a Contradiction , 2010, COLING.

[7]  Stephen Clark,et al.  Mathematical Foundations for a Compositional Distributional Model of Meaning , 2010, ArXiv.

[8]  Marco Marelli,et al.  Compositional-ly Derived Representations of Morphologically Complex Words in Distributional Semantics , 2013, ACL.

[9]  Rochelle Lieber,et al.  Handbook of Word-Formation , 2006 .

[10]  Helmut Schmidt,et al.  Probabilistic part-of-speech tagging using decision trees , 1994 .

[11]  Marco Baroni,et al.  Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space , 2010, EMNLP.

[12]  Ioannis Korkontzelos,et al.  Estimating Linear Models for Compositional Distributional Semantics , 2010, COLING.

[13]  Jan Snajder,et al.  DErivBase: Inducing and Evaluating a Derivational Morphology Resource for German , 2013, ACL.

[14]  Georgiana Dinu,et al.  DISSECT - DIStributional SEmantics Composition Toolkit , 2013, ACL.

[15]  Frans Plank,et al.  Morphologische (Ir-)Regularitäten : Aspekte der Wortstrukturtheorie , 1981 .

[16]  Jan Snajder,et al.  Derivational Smoothing for Syntactic Distributional Semantics , 2013, ACL.

[17]  Gertrud Faaß,et al.  SdeWaC - A Corpus of Parsable Sentences from the Web , 2013, GSCL.

[18]  H. H. Clark.  The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. , 1973 .

[19]  Antje Rossdeutscher,et al.  German particle verbs with 'auf': reconstructing their composition in a DRT-based framework , 2009 .

[20]  R. Harald Baayen,et al.  Predicting the dative alternation , 2007 .

[21]  Roberto Navigli,et al.  The English lexical substitution task , 2009, Lang. Resour. Evaluation.