Computational learning of construction grammars

abstract This paper presents an algorithm for learning the construction grammar of a language from a large corpus. This grammar induction algorithm has two goals: first, to show that construction grammars are learnable without highly specified innate structure; second, to develop a model of which units do or do not constitute constructions in a given dataset. The basic task of construction grammar induction is to identify the minimum set of constructions that represents the language in question with maximum descriptive adequacy. These constructions must (1) generalize across an unspecified number of units while (2) containing mixed levels of representation internally (e.g., both item-specific and schematized representations), and (3) allowing for unfilled and partially filled slots. Additionally, these constructions may (4) contain recursive structure within a given slot that needs to be reduced in order to produce a sufficiently schematic representation. In other words, these constructions are multi-length, multi-level, possibly discontinuous co-occurrences which generalize across internal recursive structures. These co-occurrences are modeled using frequency and the ΔP measure of association, expanded in novel ways to cover multi-unit sequences. This work provides important new evidence for the learnability of construction grammars as well as a tool for the automated corpus analysis of constructions.

[1]  S. Gries,et al.  Extending collostructional analysis: A corpus-based perspective on `alternations' , 2004 .

[2]  Martin Hilpert,et al.  New evidence against the modularity of grammar: Constructions, collocations, and speech perception , 2008 .

[3]  Rada Mihalcea,et al.  Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2015, North American Chapter of the Association for Computational Linguistics.

[4]  Paul Rayson,et al.  Development of the Multilingual Semantic Annotation System , 2015, NAACL.

[5]  Stefan Th. Gries,et al.  50-something years of work on collocations: What is or should be next … , 2013 .

[6]  Rens Bod,et al.  Exemplar-based syntax: How to get productivity from examples , 2006 .

[7]  Ronald W. Langacker,et al.  On the continuous debate about discreteness , 2006 .

[8]  Nick Ellis,et al.  Towards an Inventory of English Verb Argument Constructions , 2010, HLT-NAACL 2010.

[9]  Joybrato Mukherjee,et al.  Lexical gravity across varieties of English: An ICE-based study of n -grams in Asian Englishes , 2010 .

[10]  Veronika Vincze,et al.  VPCTagger: Detecting Verb-Particle Constructions With Syntax-Based Methods , 2014, MWE@EACL.

[11]  Brigham Young.  The Corpus of Contemporary American English as the first reliable monitor corpus of English , 2010 .

[12]  Ted Briscoe.  Grammatical acquisition: Inductive bias and coevolution of language and the language acquisition device , 2000 .

[13]  Willem H. Zuidema.  What are the Productive Units of Natural Language Grammar? A DOP Approach to the Automatic Identification of Constructions. , 2006, CoNLL.

[14]  Wlodek Zadrozny,et al.  NL Understanding with a Grammar of Constructions , 1994, COLING.

[15]  Carl Vogel,et al.  Proceedings of the 16th International Conference on Computational Linguistics , 1996, COLING 1996.

[16]  Jeffrey Lidz,et al.  Constructions on holiday , 2009 .

[17]  Luc Steels,et al.  Constructivist Development of Grounded Construction Grammar , 2004, ACL.

[18]  Josefina Sierra Santibáñez.  Computational Issues in Fluid Construction Grammar , 2012, Lecture Notes in Computer Science.

[19]  Stella Markantonatou,et al.  Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017) , 2017, MWE@EACL.

[20]  John Bryant,et al.  Scalable Construction-Based Parsing and Semantic Analysis , 2004, HLT-NAACL 2004.

[21]  Joachim De Beule,et al.  Computational Construction Grammar: Comparing ECG and FCG , 2012, Computational Issues in Fluid Construction Grammar.

[22]  Frederick Jelinek,et al.  Self-organizing language modeling for speech recognition , 1990 .

[23]  Markus Forsberg,et al.  From construction candidates to constructicon entries: An experiment using semi-automatic methods for identifying constructions in corpora , 2014 .

[24]  A. Goldberg.  Constructions at Work: The Nature of Generalization in Language , 2006 .

[25]  J. Silva,et al.  A Local Maxima method and a Fair Dispersion Normalization for extracting multi-word units from corpora , 2009 .

[26]  Ronald W. Langacker,et al.  Cognitive Grammar: A Basic Introduction , 2008 .

[27]  Noam Chomsky,et al.  The Logical Structure of Linguistic Theory , 1975 .

[28]  Jeffrey Heinz,et al.  Topics in Grammatical Inference , 2016, Springer Berlin Heidelberg.

[29]  Michel Achard,et al.  Language, culture and mind , 2004 .

[30]  David Wible,et al.  StringNet as a Computational Resource for Discovering and Investigating Linguistic Constructions , 2010, HLT-NAACL 2010.

[31]  Chris Biemann,et al.  Proceedings of the Workshop on Distributional Semantics and Compositionality , 2011 .

[32]  Sergei Nirenburg,et al.  Book Review: Ontological Semantics, by Sergei Nirenburg and Victor Raskin , 2004, CL.

[33]  Colin de la Higuera,et al.  Grammatical Inference for Computational Linguistics , 2015, Grammatical Inference for Computational Linguistics.

[34]  Menno van Zaanen,et al.  ABL: Alignment-Based Learning , 2000, COLING.

[35]  Milan Straka,et al.  Stop-probability estimates computed on a large corpus improve Unsupervised Dependency Parsing , 2013, ACL.

[36]  Joan L. Bybee,et al.  Language, Usage and Cognition , 2010 .

[37]  Yuji Matsumoto.  MaltParser: A language-independent system for data-driven dependency parsing , 2005 .

[38]  Roni Katzir,et al.  A cognitively plausible model for grammar induction , 2015, J. Lang. Model..

[39]  David Wible,et al.  Word similarity using constructions as contextual features , 2013, JSSP.

[40]  J. Firth,et al.  Papers in linguistics, 1934-1951 , 1957 .

[41]  Phil Blunsom,et al.  Unsupervised Induction of Tree Substitution Grammars for Dependency Parsing , 2010, EMNLP.

[42]  Mark Johnson,et al.  Improving Unsupervised Dependency Parsing with Richer Contexts and Smoothing , 2009, NAACL.

[43]  Tim van de Cruys.  Two Multivariate Generalizations of Pointwise Mutual Information , 2011, Proceedings of the Workshop on Distributional Semantics and Compositionality.

[44]  A. Goldberg.  The nature of generalization in language , 2009 .

[45]  Alex Waibel,et al.  Readings in speech recognition , 1990 .

[46]  R. Langacker.  Foundations of cognitive grammar , 1983 .

[47]  Adele E. Goldberg,et al.  Learning argument structure generalizations , 2004 .

[48]  S. Gries.  Dispersions and adjusted frequencies in corpora , 2008 .

[49]  Veronika Vincze,et al.  Dependency Parsing for Identifying Hungarian Light Verb Constructions , 2013, IJCNLP.

[50]  Valentin I. Spitkovsky,et al.  Breaking Out of Local Optima with Count Transforms and Model Recombination: A Study in Grammar Induction , 2013, EMNLP.

[51]  Vidas Daudaravicius,et al.  Gravity Counts for the boundaries of collocations , 2004 .

[52]  Simon Dennis,et al.  An exemplar-based approach to unsupervised parsing , 2005 .

[53]  Stefan Th. Gries,et al.  Collostructions: Investigating the interaction of words and constructions , 2003 .

[54]  John Goldsmith,et al.  An algorithm for the unsupervised learning of morphology , 2006, Natural Language Engineering.

[55]  Eytan Ruppin,et al.  Unsupervised learning of natural languages , 2006 .

[56]  Silvia Bernardini,et al.  The WaCky wide web: a collection of very large linguistically processed web-crawled corpora , 2009, Lang. Resour. Evaluation.

[57]  Stefan Th. Gries,et al.  Frequencies, probabilities, and association measures in usage-/exemplar-based linguistics: Some necessary clarifications , 2012 .

[58]  Alexander Clark.  Unsupervised induction of stochastic context-free grammars using distributional clustering , 2001, CoNLL.

[59]  Joan L. Bybee,et al.  From Usage to Grammar: The Mind's Response to Repetition , 2007 .

[60]  C. Fillmore,et al.  Grammatical constructions and linguistic generalizations: The What's X doing Y? construction , 1999 .

[61]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[62]  M. Tomasello.  Constructing a Language , 2005 .

[63]  Luc Steels,et al.  Design Methods for Fluid Construction Grammar , 2012, Computational Issues in Fluid Construction Grammar.

[64]  Helmut Schmidt,et al.  Probabilistic part-of-speech tagging using decision trees , 1994 .

[65]  Dan Klein,et al.  A Generative Constituent-Context Model for Improved Grammar Induction , 2002, ACL.

[66]  John A. Goldsmith,et al.  Unsupervised Learning of the Morphology of a Natural Language , 2001, CL.

[67]  Jingjie Li,et al.  A new computing method for extracting contiguous phraseological sequences from academic text corpora , 2013 .

[68]  Charles J. Fillmore,et al.  The Mechanisms of “Construction Grammar” , 1988 .

[69]  Johanna D. Moore,et al.  Proceedings of the 28th Annual Conference of the Cognitive Science Society , 2005 .

[70]  Michael Levison,et al.  The Semantic Representation of Natural Language , 2012 .