Do Grammars Minimize Dependency Length?

A well-established principle of language is that there is a preference for closely related words to be close together in the sentence. This can be expressed as a preference for dependency length minimization (DLM). In this study, we explore quantitatively the degree to which natural languages reflect DLM. We extract the dependencies from natural language text and reorder the words in such a way as to minimize dependency length. Comparing the original text with these optimal linearizations (and also with random linearizations) reveals the degree to which natural language minimizes dependency length. Tests on English data show that English shows a strong effect of DLM, with dependency length much closer to optimal than to random; the optimal English grammar also has many specific features in common with English. In German, too, dependency length is significantly less than random, but the effect is much weaker than in English. We conclude by speculating about some possible reasons for this difference between English and German.

[1]  Noah A. Smith,et al.  Parsing with Soft and Hard Constraints on Dependency Length , 2005 .

[2]  Kris Heylen.  A quantitative corpus study of German word order variation , 2004 .

[3]  Sabine Brants,et al.  The TIGER Treebank , 2001 .

[4]  Eugene Charniak,et al.  Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking , 2005, ACL.

[5]  Igor Mel’čuk,et al.  Dependency Syntax: Theory and Practice , 1987 .

[6]  M. Just,et al.  Individual differences in syntactic processing: The role of working memory , 1991 .

[7]  Barry K. Rosen,et al.  Syntactic Complexity , 1974, Inf. Control..

[8]  P. Hopper.  The syntax of the simple sentence in proto-germanic , 1975 .

[9]  Andrew Radford,et al.  Syntactic Theory and the Structure of English: Index , 1997 .

[10]  Jennifer E. Arnold,et al.  Heaviness vs. newness: The effects of structural complexity and discourse status on constituent ordering , 2015 .

[11]  E. Gibson.  Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[12]  Amit Dubey,et al.  Statistical parsing for German: modeling syntactic properties and annotation differences , 2005 .

[13]  A. Radford.  Syntactic Theory and the Structure of English: Head movement , 1997 .

[14]  John A. Hawkins,et al.  A Performance Theory of Order and Constituency , 1995 .

[15]  Lyn Frazier,et al.  Natural language parsing: Syntactic complexity , 1985 .

[16]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[17]  G. Kempen,et al.  Comparing linguistic judgments and corpus frequencies as windows on grammatical competence: A study of argument linearization in German clauses , 2008 .

[18]  Michael Don Palmer,et al.  Reflections on language , 1977 .

[19]  Noam Chomsky.  Three Factors in Language Design , 2005, Linguistic Inquiry.

[20]  Gert Webelhuth,et al.  Principles and Parameters of Syntactic Saturation , 1992 .

[21]  E. Gibson.  The dependency locality theory: A distance-based theory of linguistic complexity. , 2000 .

[22]  Ann Bies,et al.  The Penn Treebank: Annotating Predicate Argument Structure , 1994, HLT.

[23]  John Algeo,et al.  Problems in the origins and development of the English language , 1964 .

[24]  Josef van Genabith,et al.  Why is it so difficult to compare treebanks? TIGER and TüBa-D/Z revisited , 2007 .

[25]  C. Meyer.  English Corpus Linguistics An Introduction , 2002 .

[26]  T. Givon.  Topic Continuity in Discourse , 1983 .

[27]  Simon Kirby,et al.  Function, Selection, and Innateness: The Emergence of Language Universals , 1999 .

[28]  M. Dryer.  The Greenbergian word order correlations , 1992 .

[29]  Thomas Wasow,et al.  Remarks on grammatical weight , 1997, Language Variation and Change.

[30]  F. Chang,et al.  “Long before short” preference in the production of a head-final language , 2001, Cognition.

[31]  O. Gensler.  On Reconstructing the Syntagm S-Aux-O-V-Other to Proto-Niger-Congo , 1994 .

[32]  Morten H. Christiansen,et al.  Language as shaped by the brain. , 2008, The Behavioral and brain sciences.

[33]  J. Rijkhoff,et al.  Explaining word order in the noun phrase , 1990 .

[34]  Stanley Peters,et al.  Cross-Serial Dependencies in Dutch , 1982 .

[35]  Joan L. Bybee,et al.  Frequency of Use and the Organization of Language , 2006 .

[36]  Jeanette K. Gundel.  Universals of topic-comment structure , 1988 .

[37]  R. Ebert.  Historische Syntax des Deutschen , 1978 .

[38]  Jennifer E. Arnold,et al.  The Concomitant Effects of Phrase Length and Informational Content in Sentence Comprehension , 2000, Journal of psycholinguistic research.

[39]  G. Miller,et al.  Cognitive science. , 1981, Science.

[40]  Mark Steedman,et al.  Dependency and Coordination in the Grammar of Dutch and English , 1985 .

[41]  Erhard W. Hinrichs,et al.  The Tüba-D/Z Treebank: Annotating German with a Context-Free Backbone , 2004, LREC.

[42]  David Temperley,et al.  Minimization of dependency length in written English , 2007, Cognition.

[43]  L Konieczny,et al.  Locality and Parsing Complexity , 2000, Journal of psycholinguistic research.

[44]  Angela D. Friederici,et al.  Grammar overrides frequency: evidence from the online processing of flexible word order , 2002, Cognition.

[45]  Daniel Gildea,et al.  Optimizing Grammars for Minimum Dependency Length , 2007, ACL.

[46]  J. Hawkins.  Efficiency and complexity in grammars , 2004 .

[47]  David Temperley,et al.  Dependency-length minimization in natural and artificial languages* , 2008, J. Quant. Linguistics.

[48]  Douglas Biber,et al.  Representativeness in corpus design , 1993 .

[49]  S. Levinson.  Presumptive Meanings: The theory of generalized conversational implicature , 2001 .

[50]  David R. Dowty,et al.  Natural Language Parsing: Psychological, Computational, and Theoretical Perspectives , 1985 .

[51]  Jennifer Smith,et al.  An Historical Study of English: Function, Form and Change , 1998 .