On language 'utility': processing complexity and communicative efficiency.

Functionalist typologists have long argued that pressures associated with language usage influence the distribution of grammatical properties across the world's languages. Specifically, grammatical properties may be observed more often across languages because they improve a language's utility or decrease its complexity. While this approach to the study of typology offers the potential of explaining grammatical patterns in terms of general principles rather than domain-specific constraints, the notions of utility and complexity are more often grounded in intuition than empirical findings. A suitable empirical foundation might be found in the terms of processing preferences: in that case, psycholinguistic measures of complexity are then expected correlate with typological patterns. We summarize half a century of psycholinguistic work on 'processing complexity' in an attempt to make this work accessible to a broader audience: What makes something hard to process for comprehenders, and what determines speakers' preferences in production? We also briefly discuss recently emerging approaches that link preferences in production to communicative efficiency. These approaches can be seen as providing well-defined measures of utility. With these psycholinguistic findings in mind, it is possible to investigate the extent to which language usage is reflected in typological patterns. We close with a summary of paradigms that allow the link between language usage and typology to be studied empirically. WIREs Cogni Sci 2011 2 323-335 DOI: 10.1002/wcs.126 For further resources related to this article, please visit the WIREs website.

[1]  Weijia Ni,et al.  Sidestepping garden paths: Assessing the contributions of syntax, semantics and plausibility in resolving ambiguities , 1996 .

[2]  Morten H. Christiansen,et al.  Language evolution: consensus and controversies , 2003, Trends in Cognitive Sciences.

[3]  Edward Gibson,et al.  The nature of working memory capacity in sentence comprehension : Evidence against domain-specific working memory resources , 2006 .

[4]  Jennifer E. Arnold RUNNING HEAD : AVOIDING ATTACHMENT AMBIGUITIES Avoiding Attachment Ambiguities : the Role of Constituent Ordering , 2004 .

[5]  M. Just,et al.  From the SelectedWorks of Marcel Adam Just 1992 A capacity theory of comprehension : Individual differences in working memory , 2017 .

[6]  Christopher T. Kello,et al.  Verb-specific constraints in sentence processing: separating effects of lexical preference from garden-paths. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[7]  Tessa C. Warren,et al.  The influence of referential processing on sentence complexity , 2002, Cognition.

[8]  Simon Garrod,et al.  The fitness and functionality of culturally evolved communication systems , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[9]  M. Tomasello Constructing a Language: A Usage-Based Theory of Language Acquisition , 2003 .

[10]  Martin Haspelmath,et al.  Creating economical morphosyntactic patterns in language change , 2008 .

[11]  Mirjam Ernestus,et al.  Frequency distributions of uniphones, diphones, and triphones in spontaneous speech. , 2008, The Journal of the Acoustical Society of America.

[12]  Edward Gibson,et al.  Consequences of the Serial Nature of Linguistic Input for Sentenial Complexity , 2005, Cogn. Sci..

[13]  H E Wanner,et al.  An ATN approach to comprehension , 1978 .

[14]  Barry K. Rosen,et al.  Syntactic Complexity , 1974, Inf. Control..

[15]  Kathryn Bock,et al.  Syntactic effects of information availability in sentence production , 1980 .

[16]  John Hale,et al.  Uncertainty About the Rest of the Sentence , 2006, Cogn. Sci..

[17]  Roger Ratcliff,et al.  Priming in item recognition: The organization of propositions in memory for text , 1980 .

[18]  J. Bresnan,et al.  Gradient Grammar: An Effect of Animacy on the Syntax of give in New Zealand and American English , 2008 .

[19]  Randall Hendrick,et al.  Memory-Load Interference in Syntactic Processing , 2002, Psychological science.

[20]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[21]  M. Daneman,et al.  Working memory and language comprehension: A meta-analysis , 1996, Psychonomic bulletin & review.

[22]  K. Rayner,et al.  Eye movement evidence that readers maintain and act on uncertainty about past linguistic input , 2009, Proceedings of the National Academy of Sciences.

[23]  Susan M. Garnsey,et al.  The Contributions of Verb Bias and Plausibility to the Comprehension of Temporarily Ambiguous Sentences , 1997 .

[24]  G. Altmann,et al.  Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation , 2009, Cognition.

[25]  Richard L. Lewis,et al.  Computational principles of working memory in sentence comprehension , 2006, Trends in Cognitive Sciences.

[26]  R. Knight,et al.  Separable effects of priming and imageability on word processing: an ERP study. , 2002, Brain research. Cognitive brain research.

[27]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[28]  Daniel Jurafsky,et al.  A Probabilistic Model of Lexical and Syntactic Access and Disambiguation , 1996, Cogn. Sci..

[29]  J. Meaning , Sound , and Syntax : Lexical Priming in Sentence Production , 2001 .

[30]  Barbara John A Thomas Lohse,et al.  Domain Minimization in English Verb-Particle Constructions , 2004 .

[31]  R. Paget The Origin of Speech , 1927, Nature.

[32]  Daniel Jurafsky,et al.  A Bayesian Model Predicts Human Parse Preference and Reading Times in Sentence Processing , 2001, NIPS.

[33]  Amy Perfors,et al.  Simulated Evolution of Language: a Review of the Field , 2002, J. Artif. Soc. Soc. Simul..

[34]  Wilbert Spooren,et al.  Text representation : linguistic and psycholinguistic aspects , 2001 .

[35]  Benedikt Szmrecsanyi,et al.  On operationalizing syntactic complexity , 2004 .

[36]  Richard L. Lewis,et al.  Syntactic and Positional Similarity Effects in the Processing of Japanese Embeddings , 2002 .

[37]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[38]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[39]  M. Tomasello,et al.  Early syntactic creativity: a usage-based approach. , 2003, Journal of child language.

[40]  M. Aylett,et al.  Language redundancy predicts syllabic duration and the spectral characteristics of vocalic syllable nuclei. , 2006, The Journal of the Acoustical Society of America.

[41]  M. Just,et al.  Individual differences in syntactic processing: The role of working memory , 1991 .

[42]  G. Zipf The Psycho-Biology Of Language: AN INTRODUCTION TO DYNAMIC PHILOLOGY , 1999 .

[43]  M. Tanenhaus,et al.  Acquiring and processing verb argument structure: Distributional learning in a miniature language , 2008, Cognitive Psychology.

[44]  Richard L. Lewis Interference in short-term memory: The magical number two (or three) in sentence processing , 1996, Journal of psycholinguistic research.

[45]  T. Florian Jaeger,et al.  Redundancy and reduction: Speakers manage syntactic information density , 2010, Cognitive Psychology.

[46]  K. Bock,et al.  From conceptual roles to structural relations: bridging the syntactic cleft. , 1992, Psychological review.

[47]  J. Kounios,et al.  Concreteness effects in semantic processing: ERP evidence supporting dual-coding theory. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[48]  V. Ferreira Ambiguity, Accessibility, and a Division of Labor for Communicative Success. , 2008, Learning and motivation.

[49]  Thomas L. Griffiths,et al.  Using Category Structures to Test Iterated Learning as a Method for Identifying Inductive Biases , 2008, Cogn. Sci..

[50]  Béla Bollobás,et al.  The consequences of Zipf's law for syntax and symbolic reference , 2005, Proceedings of the Royal Society B: Biological Sciences.

[51]  John Hale,et al.  The Information Conveyed by Words in Sentences , 2003, Journal of psycholinguistic research.

[52]  Zenzi M. Griffin,et al.  Properties of Spoken Language Production , 2006 .

[53]  ChoiHyeWon Length and Order: A Corpus Study of Korean Dative-Accusative Construction , 2007 .

[54]  Mira Ariel Accessibility theory: An overview , 2001 .

[55]  John A. Hawkins,et al.  A Performance Theory of Order and Constituency , 1995 .

[56]  Talmy Givón On interpreting text-distributional correlations: some methodological issues , 1992 .

[57]  B. MacWhinney,et al.  Functionalism and the competition model , 1989 .

[58]  Susan M. Garnsey,et al.  Knowledge of Grammar, Knowledge of Usage: Syntactic Probabilities Affect Pronunciation Variation , 2004 .

[59]  George Kingsley Zipf,et al.  Human behavior and the principle of least effort , 1949 .

[60]  S. Goldin-Meadow,et al.  The natural order of events: How speakers of different languages represent events nonverbally , 2008, Proceedings of the National Academy of Sciences.

[61]  Mercè Prat-Sala,et al.  Discourse constraints on syntactic processing in language production , 2000 .

[62]  E. Gibson The dependency locality theory: A distance-based theory of linguistic complexity. , 2000 .

[63]  Marily Ford,et al.  A method for obtaining measures of local parsing complexity throughout sentences , 1983 .

[64]  Thomas Givon,et al.  Markedness in Grammar: Distributional, Communicative and Cognitive Correlates of Syntactic Structure , 1991 .

[65]  Eric Wanner,et al.  Language acquisition: the state of the art , 1982 .

[66]  G. Altmann,et al.  Incremental interpretation at verbs: restricting the domain of subsequent reference , 1999, Cognition.

[67]  Claude E. Shannon,et al.  A Mathematical Theory of Communications , 1948 .

[68]  R. Shillcock,et al.  Low-level predictive inference in reading: the influence of transitional probabilities on eye movements , 2003, Vision Research.

[69]  B. Velichkovsky,et al.  Eye typing in application: A comparison of two interfacing systems with ALS patients , 2008 .

[70]  W. Bruce Croft,et al.  Lexical rules vs. constructions: A false dichotomy , 2003 .

[71]  Daniel Gildea,et al.  Optimizing Grammars for Minimum Dependency Length , 2007, ACL.

[72]  F. Chang,et al.  “Long before short” preference in the production of a head-final language , 2001, Cognition.

[73]  Julian M. Pine,et al.  Constructing a Language: A Usage-Based Theory of Language Acquisition. , 2004 .

[74]  Michael Gasser,et al.  The Origins of Arbitrariness in Language , 2004 .

[75]  M. MacDonald,et al.  Individual Differences and Probabilistic Constraints in Syntactic Ambiguity Resolution , 1995 .

[76]  Ezra Van Everbroeck,et al.  Language type frequency and learnability from a connectionist perspective , 2003 .

[77]  Frank Keller,et al.  Data from eye-tracking corpora as evidence for theories of syntactic processing complexity , 2008, Cognition.

[78]  J. Kimball Seven principles of surface structure parsing in natural language , 1973 .

[79]  Stefan L. Frank,et al.  Surprisal-based comparison between a symbolic and a connectionist model of sentence processing , 2009 .

[80]  J. K. Bock,et al.  Conceptual accessibility and syntactic structure in sentence formulation , 1985, Cognition.

[81]  J. Hayes Cognition and the development of language , 1970 .

[82]  Eugene Charniak,et al.  Variation of Entropy and Parse Trees of Sentences as a Function of the Sentence Number , 2003, EMNLP.

[83]  Kathryn Bock,et al.  Prototypicality in sentence production , 2008, Cognitive Psychology.

[84]  B. McElree,et al.  Retrieval interference in sentence comprehension. , 2006, Journal of memory and language.

[85]  C. Clifton,et al.  Syntactic prediction in language comprehension: evidence from either...or. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[86]  Carlos Gómez Gallo,et al.  Incremental Syntactic Planning across Clauses , 2008 .

[87]  J. Aitchinson The crosslinguistic study of sentence processing , 1993, Journal of Child Language.

[88]  Hye-Won Choi Length and Order: A Corpus Study of Korean Dative-Accusative Construction , 2007 .

[89]  G. Kempen,et al.  A corpus study into word order variation in German subordinate clauses: Animacy affects linearization independently of grammatical function assignment , 2003 .

[90]  David I. Beaver,et al.  Lexical Variation in Relativizer Frequency , 2009 .

[91]  Reinhold Kliegl,et al.  Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus , 2008, Journal of Eye Movement Research.

[92]  M. Haspelmath,et al.  Frequency vs. iconicity in explaining grammatical asymmetries , 2008 .

[93]  Karin Harbusch,et al.  A corpus study into word order variation in German subordinate clauses: Animacy affects linearization independently of function assignment , 2003 .

[94]  Thomas L. Griffiths,et al.  The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning , 2009, Cognition.

[95]  M. H. Kelly,et al.  Prototypicality in a Linguistic Context: Effects on Sentence Structure , 1986 .

[96]  Simon Kirby,et al.  Function, Selection, and Innateness: The Emergence of Language Universals , 1999 .

[97]  P. Gordon,et al.  Similarity-based interference during language comprehension: Evidence from eye tracking during reading. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[98]  P. Gordon,et al.  Effects of noun phrase type on sentence complexity , 2004 .

[99]  John A. Hawkins,et al.  Why are categories adjacent , 2001 .

[100]  Hiroko Yamashita,et al.  Scrambled sentences in Japanese : Linguistic properties and motivations for production , 2002 .

[101]  Morten H. Christiansen,et al.  Recursive Inconsistencies Are Hard to Learn: A Connectionist Perspective on Universal Word Order Correlations , 2012 .

[102]  Alice Turk,et al.  The Smooth Signal Redundancy Hypothesis: A Functional Explanation for Relationships between Redundancy, Prosodic Prominence, and Duration in Spontaneous Speech , 2004, Language and speech.

[103]  Victor S Ferreira,et al.  Given-New Ordering Effects on the Production of Scrambled Sentences in Japanese , 2003, Journal of psycholinguistic research.

[104]  Roger Levy,et al.  Speakers optimize information density through syntactic reduction , 2006, NIPS.

[105]  Richard L. Lewis,et al.  An Activation-Based Model of Sentence Processing as Skilled Memory Retrieval , 2005, Cogn. Sci..

[106]  Louis C. W. Pols,et al.  How efficient is speech , 2003 .

[107]  Jennifer E. Arnold,et al.  Heaviness vs. newness: The effects of structural complexity and discourse status on constituent ordering , 2015 .

[108]  Joan L. Bybee,et al.  Mechanisms of Change in Grammaticization: The Role of Frequency , 2008 .

[109]  P. Gordon,et al.  Memory interference during language processing. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[110]  Susan M. Garnsey,et al.  Semantic Influences On Parsing: Use of Thematic Role Information in Syntactic Ambiguity Resolution , 1994 .

[111]  E. Gibson Linguistic complexity: locality of syntactic dependencies , 1998, Cognition.

[112]  Simon Kirby,et al.  Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language , 2008, Proceedings of the National Academy of Sciences.

[113]  Steven T. Piantadosi,et al.  The communicative function of ambiguity in language , 2011, Cognition.

[114]  Gerhard Jäger,et al.  Priming and unidirectional language change , 2008 .

[115]  Janet D. Fodor,et al.  The sausage machine: A new two-stage parsing model , 1978, Cognition.

[116]  Joan L. Bybee,et al.  Frequency and the emergence of linguistic structure , 2001 .

[117]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[118]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[119]  Dan Jurafsky,et al.  Probabilistic Modeling in Psycholinguistics: Linguistic Comprehension and Production , 2006 .

[120]  Joan L. Bybee,et al.  Frequency of Use and the Organization of Language , 2006 .

[121]  M. Corley,et al.  The Influence of Lexical , Conceptual and Planning Based Factors on Disfluency Production , 2006 .

[122]  Michael K. Tanenhaus,et al.  Producing Less Preferred Structures: More Gestures, Less Fluency , 2009 .

[123]  Lyn Frazier,et al.  Natural language parsing: Syntactic complexity , 1985 .

[124]  Thomas Wasow,et al.  Processing as a Source of Accessibility Effects on Variation , 2005 .

[125]  Richard L. Lewis,et al.  Argument-Head Distance and Processing Complexity: Explaining both Locality and Antilocality Effects , 2006 .

[126]  Morten H. Christiansen,et al.  Language as shaped by the brain. , 2008, The Behavioral and brain sciences.

[127]  Andreas Stolcke,et al.  Word predictability after hesitations: a corpus-based study , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[128]  Paul J. Hopper,et al.  Introduction to frequency and the emergence of linguistic structure , 2001 .

[129]  Mats Wirén,et al.  Syntactic Parsing , 2010, Handbook of Natural Language Processing.

[130]  Austin F. Frank,et al.  Speaking Rationally: Uniform Information Density as an Optimal Strategy for Language Production , 2008 .

[131]  Kathryn Bock,et al.  Language production : Grammatical encoding , 1994 .

[132]  D. Slobin Cognitive prerequisites for the development of grammar , 1973 .

[133]  R. Harald Baayen,et al.  Predicting the dative alternation , 2007 .

[134]  David R. Dowty,et al.  Natural Language Parsing: Psychological, Computational, and Theoretical Perspectives , 1985 .

[135]  B. MacWhinney,et al.  Sentential devices for conveying givenness and newness: A cross-cultural developmental study , 1978 .

[136]  Christopher D. Manning,et al.  Probabilistic models of word order and syntactic discontinuity , 2005 .

[137]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[138]  B. MacWhinney,et al.  The Crosslinguistic Study of Sentence Processing. , 1992 .

[139]  Ricard V. Solé,et al.  Least effort and the origins of scaling in human language , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[140]  Uriel Cohen Priva Using Information Content to PredictPhone Deletion , 2008 .

[141]  Holly P. Branigan,et al.  Contributions of animacy to grammatical function assignment and word order during production , 2008 .

[142]  John A. Hawkins,et al.  The relative order of prepositional phrases in English: Going beyond Manner–Place–Time , 1999, Language Variation and Change.

[143]  G. Dell,et al.  Effect of Ambiguity and Lexical Availability on Syntactic and Lexical Production , 2000, Cognitive Psychology.

[144]  Doris L. Payne Pragmatics of Word Order Flexibility , 1992 .

[145]  R. Ferrer i Cancho,et al.  Zipf's law from a communicative phase transition , 2005 .

[146]  Jan P. H. van Santen,et al.  Duration and spectral balance of intervocalic consonants: A case for efficient communication , 2005, Speech Commun..

[147]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[148]  H. Diessel Frequency effects in language acquisition, language use, and diachronic change , 2007 .

[149]  Morten H. Christiansen,et al.  Experience and sentence processing: Statistical learning and relative clause comprehension , 2009, Cognitive Psychology.

[150]  Jason M. Brenier,et al.  Predictability Effects on Durations of Content and Function Words in Conversational English , 2009 .

[151]  P. Schwanenflugel Why are Abstract Concepts Hard to Understand , 2013 .

[152]  Edward Gibson,et al.  The Communicative Lexicon Hypothesis , 2009 .

[153]  Thomas Wasow End-Weight from the Speaker's Perspective , 1997 .

[154]  Talmy Givón On interpreting text-distributional correlations , 1992 .

[155]  Bruno Galantucci,et al.  An Experimental Study of the Emergence of Human Communication Systems , 2005, Cogn. Sci..

[156]  John C. Trueswell,et al.  Learning to parse and its implications for language acquisition , 2007 .

[157]  Eugene Charniak,et al.  Entropy Rate Constancy in Text , 2002, ACL.

[158]  J. Hawkins Processing typology and why psychologists need to know about it , 2007 .

[159]  M. Tanenhaus Afterword The impact of “The cognitive basis for linguistic structures” , 2013 .

[160]  Edward Gibson,et al.  Effects of NP type in reading cleft sentences in English , 2005 .

[161]  J. Bresnan,et al.  Syntactic probabilities affect pronunciation variation in spontaneous speech , 2009, Language and Cognition.

[162]  T. Florian Jaeger,et al.  The Cross-linguistic Study of Sentence Production , 2009, Lang. Linguistics Compass.

[163]  Dan Jurafsky,et al.  Effects of disfluencies, predictability, and utterance position on word form variation in English conversation. , 2003, The Journal of the Acoustical Society of America.

[164]  Stephani Foraker,et al.  Memory structures that subserve sentence comprehension , 2003 .

[165]  T. Jaeger,et al.  Evidence for Efficient Language Production in Chinese , 2009 .

[166]  J. Hawkins Efficiency and complexity in grammars , 2004 .