The processing of extraposed structures in English

In most languages, most of the syntactic dependency relations found in any given sentence are projective: the word-word dependencies in the sentence do not cross each other. Some syntactic dependency relations, however, are non-projective: some of their word-word dependencies cross each other. Non-projective dependencies are both rarer and more computationally complex than projective dependencies; hence, it is of natural interest to investigate whether there are any processing costs specific to non-projective dependencies, and whether factors known to influence processing of projective dependencies also affect non-projective dependency processing. We report three self-paced reading studies, together with corpus and sentence completion studies, investigating the comprehension difficulty associated with the non-projective dependencies created by the extraposition of relative clauses in English. We find that extraposition over either verbs or prepositional phrases creates comprehension difficulty, and that this difficulty is consistent with probabilistic syntactic expectations estimated from corpora. Furthermore, we find that manipulating the expectation that a given noun will have a postmodifying relative clause can modulate and even neutralize the difficulty associated with extraposition. Our experiments rule out accounts based purely on derivational complexity and/or dependency locality in terms of linear positioning. Our results demonstrate that comprehenders maintain probabilistic syntactic expectations that persist beyond projective-dependency structures, and suggest that it may be possible to explain observed patterns of comprehension difficulty associated with extraposition entirely through probabilistic expectations.

[1]  Christopher Culy,et al.  The complexity of the vocabulary of Bambara , 1985 .

[2]  Kai-Uwe Von Fintel,et al.  Restrictions on quantifier domains , 1994 .

[3]  R. Ferrer i Cancho Why do syntactic links not cross , 2006 .

[4]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[5]  Fernando Pereira,et al.  Non-Projective Dependency Parsing using Spanning Tree Algorithms , 2005, HLT.

[6]  John Hale,et al.  The Information Conveyed by Words in Sentences , 2003, Journal of psycholinguistic research.

[7]  E. Gibson The dependency locality theory: A distance-based theory of linguistic complexity. , 2000 .

[8]  P. Gordon,et al.  Similarity-based interference during language comprehension: Evidence from eye tracking during reading. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[9]  Los Angeles,et al.  Generating Copies: An investigation into structural identity in language and grammar , 2006 .

[10]  David Temperley,et al.  Minimization of dependency length in written English , 2007, Cognition.

[11]  Roger Levy,et al.  Deep Dependencies from Context-Free Statistical Parsers: Correcting the Surface Dependency Approximation , 2004, ACL.

[12]  Edward Gibson,et al.  A computational theory of human linguistic processing: memory limitations and processing breakdown , 1991 .

[13]  菅山 謙正,et al.  Word Grammar 理論の研究 , 2005 .

[14]  Martin Kay,et al.  Syntactic Process , 1979, ACL.

[15]  P. Gordon,et al.  Effects of noun phrase type on sentence complexity , 2004 .

[16]  D. Slobin Grammatical transformations and sentence comprehension in childhood and adulthood , 1966 .

[17]  O. Krylova,et al.  Word order in Russian , 1988 .

[18]  David J. Weir,et al.  The convergence of mildly context-sensitive grammar formalisms , 1990 .

[19]  Daniel Gildea,et al.  Optimizing Grammars for Minimum Dependency Length , 2007, ACL.

[20]  Ken Hale,et al.  Warlpiri and the grammar of non-configurational languages , 1983 .

[21]  J. Bresnan,et al.  Non-configurationality in Australian aboriginal languages , 1996 .

[22]  M W Crocker,et al.  Wide-Coverage Probabilistic Sentence Processing , 2000, Journal of psycholinguistic research.

[23]  Michael K. Tanenhaus,et al.  Verb Argument Structure in Parsing and Interpretation: Evidence from wh-Questions , 1995 .

[24]  Brian Roark,et al.  Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing , 2009, EMNLP.

[25]  Leslie G. Valiant,et al.  General Context-Free Recognition in Less than Cubic Time , 1975, J. Comput. Syst. Sci..

[26]  J. Chimka Categorical Data Analysis, Second Edition , 2003 .

[27]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[28]  Jennifer E. Arnold,et al.  Heaviness vs. newness: The effects of structural complexity and discourse status on constituent ordering , 2015 .

[29]  Richard L. Lewis,et al.  Computational principles of working memory in sentence comprehension , 2006, Trends in Cognitive Sciences.

[30]  Robert L. Goldstone Returning to a New Home , 2005, Cogn. Sci..

[31]  P. Boersma,et al.  Empirical Tests of the Gradual Learning Algorithm , 2001, Linguistic Inquiry.

[32]  P. Gordon,et al.  Memory interference during language processing. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[33]  E. Gibson Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[34]  Michael C. Frank,et al.  Exploring Word Learning in a High-Density Longitudinal Corpus , 2009 .

[35]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[36]  William D. Marslen-Wilson,et al.  Crossed and nested dependencies in German and Dutch , 1986 .

[37]  T. Jaeger,et al.  Categorical Data Analysis: Away from ANOVAs (transformation or not) and towards Logit Mixed Models. , 2008, Journal of memory and language.

[38]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[39]  Stuart M. Shieber,et al.  Evidence against the context-freeness of natural language , 1985 .

[40]  J. Tenenbaum,et al.  Poverty of the Stimulus? A Rational Approach , 2006 .

[41]  T. Givón,et al.  English grammar : a function-based introduction , 1995 .

[42]  Wojciech Skut,et al.  Studien zur performanzorientierten Linguistik , 1998, Kognitionswissenschaft.

[43]  Thomas H. Cormen,et al.  Introduction to algorithms [2nd ed.] , 2001 .

[44]  Edward Gibson,et al.  The Interaction of Top-Down and Bottom-Up Statistics in the Resolution of Syntactic Category Ambiguity. , 2006 .

[45]  Edward Gibson,et al.  Consequences of the Serial Nature of Linguistic Input for Sentenial Complexity , 2005, Cogn. Sci..

[46]  David I. Beaver,et al.  Lexical Variation in Relativizer Frequency , 2009 .

[47]  Jane Simpson,et al.  Aspects of Warlpiri morphology and syntax , 1983 .

[48]  Reinhold Kliegl,et al.  Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus , 2008, Journal of Eye Movement Research.

[49]  M. Pickering,et al.  Plausibility and the Processing of Unbounded Dependencies:An Eye-Tracking Study , 1996 .

[50]  Mark Johnson,et al.  PCFG Models of Linguistic Tree Representations , 1998, CL.

[51]  Eric Sven Ristad The language complexity game , 1993 .

[52]  Lyn Frazier,et al.  Sentence processing: A tutorial review. , 1987 .

[53]  Douglas Rohde,et al.  Linger - a flexible platform for language processing experiments , 2003 .

[54]  Ellen F. Lau,et al.  The role of structural prediction in rapid syntactic analysis , 2006, Brain and Language.

[55]  Daniel Jurafsky,et al.  A Bayesian Model Predicts Human Parse Preference and Reading Times in Sentence Processing , 2001, NIPS.

[56]  Rebecca J. Panagos Meaningful Differences in the Everyday Experience of Young American Children , 1998 .

[57]  M. Coltheart Attention and Performance XII: The Psychology of Reading , 1987 .

[58]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[59]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[60]  John A. Goldsmith,et al.  Unsupervised Learning of the Morphology of a Natural Language , 2001, CL.

[61]  S. Suter Meaningful differences in the everyday experience of young American children , 2005, European Journal of Pediatrics.

[62]  Joakim Nivre,et al.  Mildly Non-Projective Dependency Structures , 2006, ACL.

[63]  Marc Brysbaert,et al.  Exposure-based models of human parsing: Evidence for the use of coarse-grained (nonlexical) statistical records , 1995 .

[64]  J. Fodor,et al.  The Psychology of Language: An Introduction to Psycholinguistics and Generative Grammar , 1976 .

[65]  Alexis Manaster Ramer Review of The language complexity game by Eric Sven Ristad. The MIT Press 1993. , 1995 .

[66]  T. Mexia,et al.  Author ' s personal copy , 2009 .

[67]  Geoffrey K. Pullum,et al.  Generalized Phrase Structure Grammar , 1985 .

[68]  Gabriella Hermon,et al.  Extraposition and the complement principle , 1990 .

[69]  Geoffrey J. Huck,et al.  Extraposition and focus , 1990 .

[70]  John A. Hawkins,et al.  A Performance Theory of Order and Constituency , 1995 .

[71]  G. Seth Psychology of Language , 1968, Nature.

[72]  Roger Levy,et al.  Speakers optimize information density through syntactic reduction , 2006, NIPS.

[73]  J. Pennebaker,et al.  Are Women Really More Talkative Than Men? , 2007, Science.

[74]  Hans C. Jessen,et al.  Applied Logistic Regression Analysis , 1996 .

[75]  Refractor Vision , 2000, The Lancet.

[76]  K. Rayner,et al.  Contextual effects on word perception and eye movements during reading , 1981 .

[77]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[78]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[79]  Victor H. Yngve,et al.  A model and an hypothesis for language structure , 1960 .

[80]  Michael J. Spivey,et al.  Syntactic ambiguity resolution in discourse: modeling the effects of referential context and lexical frequency. , 1998, Journal of experimental psychology. Learning, memory, and cognition.

[81]  C. Clifton,et al.  Syntactic prediction in language comprehension: evidence from either...or. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[82]  Philip Resnik,et al.  Probabilistic Tree-Adjoining Grammar as a Framework for Statistical Natural Language Processing , 1992, COLING.

[83]  Mark-Jan Nederhof,et al.  The Computational Complexity of the Correct-Prefix Property for TAGs , 1999, Comput. Linguistics.

[84]  Lyn Frazier,et al.  Heavy NP shift is the parser's last resort: Evidence from eye movements. , 2006, Journal of memory and language.

[85]  M. Tanenhaus,et al.  Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension , 1998 .

[86]  Dedre Gentner,et al.  Relations, Objects, and the Composition of Analogies , 2006, Cogn. Sci..

[87]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[88]  Thomas Wasov,et al.  Postverbal behavior , 2002, CSLI lecture notes series.

[89]  Richard L. Lewis,et al.  An Activation-Based Model of Sentence Processing as Skilled Memory Retrieval , 2005, Cogn. Sci..

[90]  E. Keenan,et al.  Noun Phrase Accessibility and Universal Grammar , 2008 .

[91]  J. Hawkins Efficiency and complexity in grammars , 2004 .

[92]  P. Good Permutation, Parametric, and Bootstrap Tests of Hypotheses , 2005 .

[93]  G. Miller Some psychological studies of grammar. , 1962 .

[94]  Srini Narayanan,et al.  Bayesian Models of Human Sentence Processing , 1998 .

[95]  C. Clifton,et al.  Comprehending Sentences with Long-Distance Dependencies , 1989 .

[96]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[97]  P. Culicover,et al.  English Focus Constructions and the Theory of Grammar , 1990 .

[98]  John Hale,et al.  Uncertainty About the Rest of the Sentence , 2006, Cogn. Sci..

[99]  Edith Kaan,et al.  Cross-serial dependencies in dutch: Testing the influence of NP type on processing load , 2004, Memory & cognition.

[100]  Rens Bod,et al.  Beyond Grammar: An Experience-Based Theory of Language , 1998 .

[101]  K. Vijay-Shankar,et al.  SOME COMPUTATIONAL PROPERTIES OF TREE ADJOINING GRAMMERS , 1985, ACL 1985.

[102]  E. Ziegel Permutation, Parametric, and Bootstrap Tests of Hypotheses (3rd ed.) , 2005 .

[103]  Igor Mel’čuk,et al.  Dependency Syntax: Theory and Practice , 1987 .

[104]  Daniel Jurafsky,et al.  A Probabilistic Model of Lexical and Syntactic Access and Disambiguation , 1996, Cogn. Sci..

[105]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[106]  H. H. Clark,et al.  Audience Design in Meaning and Reference , 1982 .

[107]  Morten H. Christiansen,et al.  Reassessing working memory: comment on Just and Carpenter (1992) and Waters and Caplan (1996). , 2002, Psychological review.

[108]  Ivan A. Sag,et al.  Book Reviews: Head-driven Phrase Structure Grammar and German in Head-driven Phrase-structure Grammar , 1996, CL.

[109]  Elaine J. Francis Grammatical weight and relative clause extraposition in English , 2010 .

[110]  S. Menard Applied Logistic Regression Analysis , 1996 .

[111]  T. Griffiths,et al.  A Bayesian framework for word segmentation: Exploring the effects of context , 2009, Cognition.

[112]  L Konieczny,et al.  Locality and Parsing Complexity , 2000, Journal of psycholinguistic research.

[113]  Marcel Adam Just,et al.  A Model of the Time Course and Content of Reading , 1982, Cogn. Sci..

[114]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[115]  Carl Jesse Pollard,et al.  Generalized phrase structure grammars, head grammars, and natural language , 1984 .

[116]  Jay Earley,et al.  An efficient context-free parsing algorithm , 1970, Commun. ACM.

[117]  Benedikt Szmrecsanyi,et al.  On operationalizing syntactic complexity , 2004 .

[118]  Silvia P. Gennari,et al.  Semantic indeterminacy in object relative clauses. , 2008, Journal of memory and language.

[119]  Maryellen C. MacDonald,et al.  Linking production and comprehension processes: The case of relative clauses , 2009, Cognition.

[120]  C. Clifton,et al.  The independence of syntactic processing , 1986 .

[121]  Jennifer E. Arnold,et al.  Put in last position something previously unmentioned: Word order effects on referential expectancy and reference comprehension , 2008 .

[122]  H. Kucera,et al.  Computational analysis of present-day American English , 1967 .

[123]  Harry Bunt,et al.  Formal tools for describing and processing discontinuous constituency structure , 1996 .

[124]  B. Velichkovsky,et al.  Eye typing in application: A comparison of two interfacing systems with ALS patients , 2008 .

[125]  Aravind K. Joshi,et al.  Some Computational Properties of Tree Adjoining Grammars , 1985, ACL.

[126]  John Robert Ross,et al.  Constraints on variables in syntax , 1967 .

[127]  John J. Godfrey,et al.  SWITCHBOARD: telephone speech corpus for research and development , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[128]  Philip H. Miller,et al.  Strong generative capacity - the semantics of linguistic formalism , 2000, CSLI lecture notes series.

[129]  Janet D. Fodor,et al.  The sausage machine: A new two-stage parsing model , 1978, Cognition.

[130]  Daniel H. Younger,et al.  Recognition and Parsing of Context-Free Languages in Time n^3 , 1967, Inf. Control..

[131]  M. Brysbaert,et al.  The correspondence between sentence production and corpus frequencies in modifier attachment , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[132]  Thomas Wasow,et al.  Remarks on grammatical weight , 1997, Language Variation and Change.

[133]  Roger Levy,et al.  Tregex and Tsurgeon: tools for querying and manipulating tree data structures , 2006, LREC.

[134]  Simon Kirby,et al.  Function, Selection, and Innateness: The Emergence of Language Universals , 1999 .

[135]  Joakim Nivre,et al.  Pseudo-Projective Dependency Parsing , 2005, ACL.

[136]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[137]  Nathaniel J. Smith,et al.  Optimal Processing Times in Reading: A Formal Model and Empirical Investigation , 2008 .

[138]  Frank Keller,et al.  Data from eye-tracking corpora as evidence for theories of syntactic processing complexity , 2008, Cognition.

[139]  Aravind K. Joshi,et al.  Tree Adjunct Grammars , 1975, J. Comput. Syst. Sci..

[140]  J. Hawkins PROCESSING COMPLEXITY AND FILLER-GAP DEPENDENCIES ACROSS GRAMMARS , 1999 .

[141]  Roger Levy,et al.  Minimal-length linearizations for mildly context-sensitive dependency trees , 2009, NAACL.

[142]  Edward P. Stabler,et al.  Derivational Minimalism , 1996, LACL.