Incrementality and Prediction in Human Sentence Processing

We identify a number of principles with respect to prediction that, we argue, underpin adult language comprehension: (a) comprehension consists in realizing a mapping between the unfolding sentence and the event representation corresponding to the real-world event being described; (b) the realization of this mapping manifests as the ability to predict both how the language will unfold, and how the real-world event would unfold if it were being experienced directly; (c) concurrent linguistic and nonlinguistic inputs, and the prior internal states of the system, each drive the predictive process; (d) the representation of prior internal states across a representational substrate common to the linguistic and nonlinguistic domains enables the predictive process to operate over variable time frames and variable levels of representational abstraction. We review empirical data exemplifying the operation of these principles and discuss the relationship between prediction, event structure, thematic role assignment, and incrementality.

[1]  G. Altmann,et al.  Discourse-mediation of the mapping between language and the visual world: Eye movements and mental representation , 2009, Cognition.

[2]  Z. Dienes,et al.  Rule learning by seven-month-old infants and neural networks , 1999 .

[3]  Rolf A. Zwaan,et al.  Chapter 18 – Discourse Comprehension , 2006 .

[4]  L. Stark,et al.  Spontaneous Eye Movements During Visual Imagery Reflect the Content of the Visual Scene , 1997, Journal of Cognitive Neuroscience.

[5]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[6]  Michael P. Kaschak,et al.  Grounding language in action , 2002, Psychonomic bulletin & review.

[7]  Rajesh P. N. Rao,et al.  Spike-Timing-Dependent Hebbian Plasticity as Temporal Difference Learning , 2001, Neural Computation.

[8]  Anne Pier Salverda,et al.  The role of prosodic boundaries in the resolution of lexical embedding in speech comprehension , 2003, Cognition.

[9]  Bradley L. Pritchett Grammatical Competence and Parsing Performance , 1992 .

[10]  Colin M. Brown,et al.  Anticipating upcoming words in discourse: evidence from ERPs and reading times. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[11]  Christine D. Wilson,et al.  Grounding conceptual knowledge in modality-specific systems , 2003, Trends in Cognitive Sciences.

[12]  James L. McClelland,et al.  A computational model of semantic memory impairment: modality specificity and emergent category specificity. , 1991, Journal of experimental psychology. General.

[13]  Gerry T. M. Altmann,et al.  Thematic role assignment in context , 1999 .

[14]  T. Sejnowski,et al.  The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. , 1994, Learning & memory.

[15]  David R. Dowty,et al.  Word Meaning and Montague Grammar , 1979 .

[16]  Yuki Kamide,et al.  Now you see it, now you don't: mediating the mapping between language and the visual world , 2004 .

[17]  John Whitman,et al.  --; CHARLES CLIFTON; and JANET RANDALL. 1983. Filling gaps: Decision principles and structure in sentence comprehension. Cognition 13.187-222. , and JANET FODOR. 1978. The sausage machine: A new two-stage parsing model. Cognition , 1997 .

[18]  K. McRae,et al.  Integrating Verbs, Situation Schemas, and Thematic Role Concepts , 2001 .

[19]  Michael K. Tanenhaus,et al.  Verb Argument Structure in Parsing and Interpretation: Evidence from wh-Questions , 1995 .

[20]  Mark S. Gilzenrat,et al.  A Systems-Level Perspective on Attention and Cognitive Control: Guided Activation, Adaptive Gating, Conflict Monitoring, and Exploitation versus Exploration. , 2004 .

[21]  Jeffrey M. Zacks,et al.  Event perception: a mind-brain perspective. , 2007, Psychological bulletin.

[22]  Gerry Altmann,et al.  Mapping across Domains Without Feedback: A Neural Network Model of Transfer of Implicit Knowledge , 1999, Cogn. Sci..

[23]  R. Shillcock,et al.  Low-level predictive inference in reading: the influence of transitional probabilities on eye movements , 2003, Vision Research.

[24]  B. Laeng,et al.  Eye scanpaths during visual imagery reenact those of perception of the same visual scene , 2002 .

[25]  Z. Harris,et al.  Foundations of language , 1941 .

[26]  G. Altmann,et al.  The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing , 2007 .

[27]  Roger M. Cooper,et al.  The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. , 1974 .

[28]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[29]  Rajesh P. N. Rao,et al.  Self–organizing neural systems based on predictive learning , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[30]  Michael J Cortese,et al.  Handbook of Psycholinguistics , 2011 .

[31]  John Morton,et al.  The Effects of Context upon Speed of Reading, Eye Movements and Eye-voice Span , 1964 .

[32]  Bruno Laeng,et al.  Eye scanpaths during visual imagery reenact those of perception of the same visual scene , 2002, Cogn. Sci..

[33]  Katherine A. DeLong,et al.  Probabilistic word pre-activation during language comprehension inferred from electrical brain activity , 2005, Nature Neuroscience.

[34]  Daniel C. Richardson,et al.  When facts go down the rabbit hole: Contrasting features and objecthood as indexes to memory , 2008, Cognition.

[35]  R. Shillcock,et al.  Eye Movements Reveal the On-Line Computation of Lexical Probabilities During Reading , 2003, Psychological science.

[36]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[37]  M. Tanenhaus,et al.  Subcategorical mismatches and the time course of lexical access: Evidence for lexical competition , 2001 .

[38]  Paul D. Allopenna,et al.  Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models , 1998 .

[39]  M. Posner The Cognitive Neuroscience of Attention , 2020 .

[40]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[41]  J. Morton THE EFFECTS OF CONTEXT ON THE VISUAL DURATION THRESHOLD FOR WORDS. , 1964, British journal of psychology.

[42]  Michael I. Jordan Serial Order: A Parallel Distributed Processing Approach , 1997 .

[43]  G. Bower,et al.  Temporal and spatial distance in situation models , 2000, Memory & cognition.

[44]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[45]  G. Altmann Language-mediated eye movements in the absence of a visual world: the ‘blank screen paradigm’ , 2004, Cognition.

[46]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[47]  Lila R. Gleitman,et al.  The Role of Syntax in Verb Learning , 2019, The Handbook of Child Language.

[48]  Silvia P. Gennari,et al.  Semantic indeterminacy in object relative clauses. , 2008, Journal of memory and language.

[49]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[50]  Fernanda Ferreira,et al.  Scene Perception for Psycholinguists. , 2004 .

[51]  E. Tulving,et al.  STIMULUS INFORMATION AND CONTEXTUAL INFORMATION AS DETERMINANTS OF TACHISTOSCOPIC RECOGNITION OF WORDS. , 1963, Journal of experimental psychology.

[52]  John C. Trueswell,et al.  The role of thematic structures in interpretation and parsing. , 1989 .

[53]  L. Gleitman The Structural Sources of Verb Meanings , 2020, Sentence First, Arguments Afterward.

[54]  Julie C. Sedivy,et al.  Eye movements as a window into real-time spoken language comprehension in natural contexts , 1995, Journal of psycholinguistic research.

[55]  Mark S. Seidenberg,et al.  On the nature and scope of featural representations of word meaning. , 1997, Journal of experimental psychology. General.

[56]  S Carlota,et al.  Dowty, David. 1979. Word Meaning and Montague Grammar: The Semantics of Verbs and Times in Generative Semantics and in Montague's PTQ. Dordrecht: Reidel. , 2005 .

[57]  E. Wanner The ATN and the sausage machine: Which one is baloney? , 1980, Cognition.

[58]  J. Elman,et al.  A basis for generating expectancies for verbs from nouns , 2005, Memory & cognition.

[59]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[60]  Matthew W. Crocker,et al.  The Coordinated Interplay of Scene, Utterance, and World Knowledge: Evidence From Eye Tracking , 2006, Cogn. Sci..

[61]  A. Glenberg,et al.  Mental models contribute to foregrounding during text comprehension , 1987 .

[62]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[63]  G. Altmann Learning and development in neural networks – the importance of prior experience , 2002, Cognition.

[64]  Daniel C. Richardson,et al.  Representation, space and Hollywood Squares: looking at things that aren't there anymore , 2000, Cognition.

[65]  Gerry T. M. Altmann,et al.  The Ascent of Babel: An Exploration of Language, Mind, and Understanding , 1997 .

[66]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .

[67]  Dan Jurafsky,et al.  Probabilistic Modeling in Psycholinguistics: Linguistic Comprehension and Production , 2006 .

[68]  Todd R. Ferretti,et al.  Thematic Roles as Verb-specific Concepts , 1997 .

[69]  M. Tanenhaus,et al.  Gradient effects of within-category phonetic variation on lexical access , 2002, Cognition.

[70]  L. Tyler,et al.  Towards a distributed account of conceptual knowledge , 2001, Trends in Cognitive Sciences.

[71]  Rolf A. Zwaan,et al.  Discourse comprehension. , 1997, Annual review of psychology.

[72]  Michael J. Spivey,et al.  Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects , 2001, Psychological research.

[73]  Mark S. Seidenberg,et al.  Computing the meanings of words in reading: cooperative division of labor between visual and phonological processes. , 2004, Psychological review.

[74]  Janet D. Fodor,et al.  The sausage machine: A new two-stage parsing model , 1978, Cognition.

[75]  G. Altmann,et al.  Word meaning and the control of eye fixation: semantic competitor effects and the visual world paradigm , 2005, Cognition.

[76]  Matthew W. Crocker,et al.  The influence of recent scene events on spoken comprehension: Evidence from eye movements , 2007 .

[77]  G. Kane Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, vol 2: Psychological and Biological Models , 1994 .

[78]  M. Tanenhaus,et al.  Actions and affordances in syntactic ambiguity resolution. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[79]  Marshall R. Mayberry,et al.  Learning to Attend: A Connectionist Model of Situated Language Comprehension , 2009, Cogn. Sci..

[80]  I. Fischler,et al.  Automatic and attentional processes in the effects of sentence contexts on word recognition , 1979 .

[81]  G. Altmann,et al.  Incremental interpretation at verbs: restricting the domain of subsequent reference , 1999, Cognition.

[82]  三嶋 博之 The theory of affordances , 2008 .

[83]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[84]  Jeffrey M. Zacks,et al.  A Computational Model of Event Segmentation From Perceptual Prediction , 2007, Cogn. Sci..

[85]  Karl G. D. Bailey,et al.  Good-Enough Representations in Language Comprehension , 2002 .

[86]  J. Elman Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[87]  Z. Harris,et al.  Foundations of Language , 1940 .

[88]  W. Chafe Givenness, contrastiveness, definiteness, subjects, topics, and point of view , 1976 .