A Tale of Two Positivities and the N400: Distinct Neural Signatures Are Evoked by Confirmed and Violated Predictions at Different Levels of Representation

It has been proposed that hierarchical prediction is a fundamental computational principle underlying neurocognitive processing. Here, we ask whether the brain engages distinct neurocognitive mechanisms in response to inputs that fulfill versus violate strong predictions at different levels of representation during language comprehension. Participants read three-sentence scenarios in which the third sentence constrained for a broad event structure, for example, {Agent caution animate–Patient}. High constraint contexts additionally constrained for a specific event/lexical item, for example, a two-sentence context about a beach, lifeguards, and sharks constrained for the event, {Lifeguards cautioned Swimmers}, and the specific lexical item swimmers. Low constraint contexts did not constrain for any specific event/lexical item. We measured ERPs on critical nouns that fulfilled and/or violated each of these constraints. We found clear, dissociable effects to fulfilled semantic predictions (a reduced N400), to event/lexical prediction violations (an increased late frontal positivity), and to event structure/animacy prediction violations (an increased late posterior positivity/P600). We argue that the late frontal positivity reflects a large change in activity associated with successfully updating the comprehender's current situation model with new unpredicted information. We suggest that the late posterior positivity/P600 is triggered when the comprehender detects a conflict between the input and her model of the communicator and communicative environment. This leads to an initial failure to incorporate the unpredicted input into the situation model, which may be followed by second-pass attempts to make sense of the discourse through reanalysis, repair, or reinterpretation. Together, these findings provide strong evidence that confirmed and violated predictions at different levels of representation manifest as distinct spatiotemporal neural signatures.

[1]  Gina R. Kuperberg,et al.  Neural mechanisms of language comprehension: Challenges to syntax , 2007, Brain Research.

[2]  D. Caplan,et al.  The role of animacy and thematic relationships in processing active English sentences: Evidence from event-related potentials , 2007, Brain and Language.

[3]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[4]  Linda B. Smith,et al.  The Cambridge Handbook of Psycholinguistics: Developing Categories and Concepts , 2012 .

[5]  Richard N. Aslin,et al.  Learning to Represent a Multi-Context Environment: More than Detecting Changes , 2012, Front. Psychology.

[6]  Gary S Dell,et al.  The P-chain: relating sentence production and its disorders to comprehension and acquisition , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[7]  Kerstin Preuschoff,et al.  Balancing New against Old Information: The Role of Puzzlement Surprise in Learning , 2018, Neural Computation.

[8]  H. Schriefers,et al.  Prediction in language comprehension beyond specific words: An ERP study on sentence comprehension in Polish , 2013 .

[9]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[10]  Rajesh P. N. Rao,et al.  Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex , 1997, Neural Computation.

[11]  Floris P. de Lange,et al.  How Prediction Errors Shape Perception, Attention, and Motivation , 2012, Front. Psychology.

[12]  David M. Groppe,et al.  Overlapping dual ERP responses to low cloze probability sentence continuations. , 2011, Psychophysiology.

[13]  James L. McClelland,et al.  Modelling the N400 brain potential as change in a probabilistic representation of meaning , 2018, Nature Human Behaviour.

[14]  G. Kuperberg,et al.  Reversing expectations during discourse comprehension , 2015, Language, cognition and neuroscience.

[15]  J. Nicol,et al.  On the Distinctiveness, Independence, and Time Course of the Brain Responses to Syntactic and Semantic Anomalies. , 1999 .

[16]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[17]  H. Kolk,et al.  Structure and limited capacity in verbal working memory: A study with event-related potentials , 2003, Brain and Language.

[18]  C. Van Petten,et al.  Prediction during language comprehension: benefits, costs, and ERP components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[19]  Dave F. Kleinschmidt,et al.  Robust speech perception: recognize the familiar, generalize to the similar, and adapt to the novel. , 2015, Psychological review.

[20]  Steven J. Luck,et al.  ERPLAB: an open-source toolbox for the analysis of event-related potentials , 2014, Front. Hum. Neurosci..

[21]  Kara D. Federmeier Thinking ahead: the role and roots of prediction in language comprehension. , 2007, Psychophysiology.

[22]  M. Kutas,et al.  Expect the Unexpected: Event-related Brain Response to Morphosyntactic Violations , 1998 .

[23]  Gina R Kuperberg,et al.  Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events , 2016, Language, cognition and neuroscience.

[24]  Gina R Kuperberg,et al.  What do we mean by prediction in language comprehension? , 2016, Language, cognition and neuroscience.

[25]  A. Miyake,et al.  Individual Differences in Verbal Working Memory Underlie a Tradeoff Between Semantic and Structural Processing Difficulty During Language Comprehension: An ERP Investigation , 2017, Journal of experimental psychology. Learning, memory, and cognition.

[26]  Ellen F. Lau,et al.  A Direct Comparison of N400 Effects of Predictability and Incongruity in Adjective-Noun Combination , 2016 .

[27]  J. Toomasian The Case for the Case , 2016, Perfusion.

[28]  Kara D. Federmeier,et al.  Age-related and individual differences in the use of prediction during language comprehension , 2010, Brain and Language.

[29]  Andrea Weber,et al.  When One Person's Mistake Is Another's Standard Usage: The Effect of Foreign Accent on Syntactic Processing , 2012, Journal of Cognitive Neuroscience.

[30]  M. Traxler,et al.  Effects of prediction and contextual support on lexical processing: Prediction takes precedence , 2015, Cognition.

[31]  Noah D. Goodman,et al.  Wonky worlds: Listeners revise world knowledge when utterances are odd , 2015, CogSci.

[32]  Edward J. Alexander,et al.  1 Supplementary Material : What we Know about Knowing : Presuppositions generated by factive verbs influence downstream neural processing , 2018 .

[33]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[34]  M. Kutas,et al.  Anticipating Words and Their Gender: An Event-related Brain Potential Study of Semantic Integration, Gender Expectancy, and Gender Agreement in Spanish Sentence Reading , 2004, Journal of Cognitive Neuroscience.

[35]  P. Holcomb,et al.  Event-related brain potentials elicited by syntactic anomaly , 1992 .

[36]  Kara D. Federmeier,et al.  A Beautiful Day in the Neighborhood: An Event-Related Potential Study of Lexical Relationships and Prediction in Context. , 2009, Journal of memory and language.

[37]  Hartmut Fitz,et al.  Getting real about Semantic Illusions: Rethinking the functional role of the P600 in language comprehension , 2012, Brain Research.

[38]  Peter,et al.  Semantic Unification , 2008 .

[39]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[40]  Jeffrey Gruber Studies in lexical relations , 1965 .

[41]  Judith F. Kroll,et al.  Cognitive control ability mediates prediction costs in monolinguals and bilinguals , 2018, Cognition.

[42]  Colin M. Brown,et al.  The Mechanism Underlying Backward Priming in a Lexical Decision Task: Spreading Activation versus Semantic Matching , 1998 .

[43]  Dorothee J. Chwilla,et al.  Monitoring in Language Perception: Mild and Strong Conflicts Elicit Different ERP Patterns , 2010, Journal of Cognitive Neuroscience.

[44]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[45]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[46]  James L. McClelland Integrating probabilistic models of perception and interactive neural networks: a historical and tutorial review , 2013, Front. Psychol..

[47]  Rebecca Treiman,et al.  The English Lexicon Project , 2007, Behavior research methods.

[48]  Richard N. Aslin,et al.  Incremental implicit learning of bundles of statistical patterns , 2016, Cognition.

[49]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[50]  Dietmar Roehm,et al.  Semantic prediction in language comprehension: evidence from brain potentials , 2016, Language, cognition and neuroscience.

[51]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[52]  G. Dell,et al.  Becoming syntactic. , 2006, Psychological review.

[53]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[54]  Aya Meltzer-Asscher,et al.  Lexical Inhibition due to Failed Prediction: Behavioral Evidence and ERP Correlates , 2017, Journal of experimental psychology. Learning, memory, and cognition.

[55]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[56]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[57]  M. Schlesewsky,et al.  The P600-as-P3 hypothesis revisited: Single-trial analyses reveal that the late EEG positivity following linguistically deviant material is reaction time aligned , 2014, Brain and Language.

[58]  R. Jackendoff The Status of Thematic Relations in Linguistic Theory , 1987 .

[59]  Heather J. Ferguson,et al.  Dissociable effects of prediction and integration during language comprehension: Evidence from a large-scale study using brain potentials , 2018, bioRxiv.

[60]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[61]  G. Kuperberg,et al.  Multiple Influences of Semantic Memory on Sentence Processing: Distinct Effects of Semantic Relatedness on Violations of Real-World Event/State Knowledge and Animacy Selection Restrictions. , 2012, Journal of memory and language.

[62]  Charles Kemp,et al.  Bayesian models of cognition , 2008 .

[63]  Jason Bohan,et al.  Anomalies at the Borderline of Awareness: An ERP Study , 2011, Journal of Cognitive Neuroscience.

[64]  Christian J. Fiebach,et al.  Finding the P3 in the P600: Decoding shared neural mechanisms of responses to syntactic violations and oddball targets , 2019, NeuroImage.

[65]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[66]  M. Kutas In the company of other words: Electrophysiological evidence for single-word and sentence context effects , 1993 .

[67]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[68]  Jelena Mirkovic,et al.  Incrementality and Prediction in Human Sentence Processing , 2009, Cogn. Sci..

[69]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[70]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[71]  Colin M. Brown,et al.  When and how do listeners relate a sentence to the wider discourse? Evidence from the N400 effect. , 2003, Brain research. Cognitive brain research.

[72]  Colin Phillips,et al.  A “bag-of-arguments” mechanism for initial verb predictions , 2016 .

[73]  C. Gallistel,et al.  The Perception of Probability , 2022 .

[74]  J. Elman,et al.  Generalized event knowledge activation during online sentence comprehension. , 2012, Journal of memory and language.

[75]  Jerome L. Myers,et al.  Accessing the discourse representation during reading , 1998 .

[76]  Phillip J. Holcomb,et al.  Neural correlates of processing syntactic, semantic, and thematic relationships in sentences , 2006 .

[77]  Hiroko Nakano,et al.  Speech and Span: Working Memory Capacity Impacts the Use of Animacy but Not of World Knowledge during Spoken Sentence Comprehension , 2010, Journal of Cognitive Neuroscience.

[78]  Ken McRae,et al.  People Use their Knowledge of Common Events to Understand Language, and Do So as Quickly as Possible , 2009, Lang. Linguistics Compass.

[79]  Nelda Melissa Chavez Individual differences in verbal working memory, visuo-spatial working memory, and metacognition: Learning from text in a hypertext environment , 2002 .

[80]  Pienie Zwitserlood,et al.  Dissociating predictability, plausibility and possibility of sentence continuations in reading: evidence from late-positivity ERPs , 2018, PeerJ.

[81]  Katherine A. DeLong,et al.  Predictability, plausibility, and two late ERP positivities during written sentence comprehension , 2014, Neuropsychologia.

[82]  Wilson L. Taylor,et al.  “Cloze Procedure”: A New Tool for Measuring Readability , 1953 .

[83]  L. Osterhout,et al.  The independence of combinatory semantic processing: Evidence from event-related potentials , 2005 .

[84]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[85]  P. Schumacher When combinatorial processing results in reconceptualization: toward a new approach of compositionality , 2013, Front. Psychol..

[86]  D. Mumford On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[87]  D. Caplan,et al.  Electrophysiological distinctions in processing conceptual relationships within simple sentences. , 2003, Brain research. Cognitive brain research.

[88]  Ellen F. Lau,et al.  Wait a second! delayed impact of argument roles on on-line verb prediction , 2018 .

[89]  Barbara B. Levin,et al.  English verb classes and alternations , 1993 .

[90]  Michael C. Frank,et al.  Predicting Pragmatic Reasoning in Language Games , 2012, Science.

[91]  Edward J. Alexander,et al.  Neural Evidence for the Prediction of Animacy Features during Language Comprehension: Evidence from MEG and EEG Representational Similarity Analysis , 2019, The Journal of Neuroscience.

[92]  Mary Hare,et al.  Activating event knowledge , 2009, Cognition.

[93]  Michael W. Spratling Predictive coding as a model of biased competition in visual attention , 2008, Vision Research.

[94]  David R. Dowty On the Semantic Content of the Notion of ‘Thematic Role’ , 1989 .

[95]  Dorothee J. Chwilla,et al.  Monitoring in Language Perception , 2009, Lang. Linguistics Compass.

[96]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[97]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[98]  A. Friederici,et al.  Syntactic Gender and Semantic Expectancy: ERPs Reveal Early Autonomy and Late Interaction , 2000, Journal of Cognitive Neuroscience.

[99]  Mante S. Nieuwland,et al.  When Peanuts Fall in Love: N400 Evidence for the Power of Discourse , 2005, Journal of Cognitive Neuroscience.

[100]  Edward W. Wlotko,et al.  Going the Extra Mile: Effects of Discourse Context on Two Late Positivities During Language Comprehension , 2020, Neurobiology of Language.

[101]  Colin M. Brown,et al.  The syntactic positive shift (sps) as an erp measure of syntactic processing , 1993 .

[102]  T. Ditman,et al.  Electrophysiological insights into the processing of nominal metaphors , 2010, Neuropsychologia.

[103]  Laurence Aitchison,et al.  With or without you: predictive coding and Bayesian inference in the brain , 2017, Current Opinion in Neurobiology.

[104]  C. Van Petten,et al.  Lexical versus conceptual anticipation during sentence processing: frontal positivity and N400 ERP components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[105]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[106]  Gina R Kuperberg,et al.  Electrophysiological evidence for use of the animacy hierarchy, but not thematic role assignment, during verb-argument processing , 2011, Language and cognitive processes.

[107]  P. Schwanenflugel,et al.  Semantic relatedness and the scope of facilitation for upcoming words in sentences. , 1988 .

[108]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[109]  Marta Kutas,et al.  CHAPTER 15 A Look around at What Lies Ahead: Prediction and Predictability in Language Processing , 2010 .

[110]  Sandy Polishuk I Look Around , 2003 .

[111]  Pierre Baldi,et al.  Of bits and wows: A Bayesian theory of surprise with applications to attention , 2010, Neural Networks.

[112]  H. H. Clark The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. , 1973 .

[113]  Martin Paczynski,et al.  Establishing Causal Coherence across Sentences: An ERP Study , 2011, Journal of Cognitive Neuroscience.

[114]  Ellen F. Lau,et al.  Dissociating N400 Effects of Prediction from Association in Single-word Contexts , 2013, Journal of Cognitive Neuroscience.

[115]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[116]  Kara D. Federmeier,et al.  Multiple effects of sentential constraint on word processing , 2007, Brain Research.

[117]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.