A Tale of Two Positivities (and the N400): Distinct neural signatures are evoked by confirmed and violated predictions at different levels of representation

It has been proposed that hierarchical prediction is a fundamental computational principle underlying neurocognitive processing. Here we ask whether the brain engages distinct neurocognitive mechanisms in response to inputs that fulfill versus violate strong predictions at different levels of representation during language comprehension. Participants read three-sentence scenarios in which the third sentence constrained for a broad event structure, e.g. {Agent caution animate-Patient}. High constraint contexts additionally constrained for a specific event/lexical item, e.g. a two-sentence context about a beach, lifeguards and sharks constrained for the event, {Lifeguards cautioned Swimmers} and the specific lexical item, “swimmers”. Low constraint contexts did not constrain for any specific event/lexical item. We measured ERPs on critical nouns that fulfilled and/or violated each of these constraints. We found clear, dissociable effects to fulfilled semantic predictions (a reduced N400), to event/lexical prediction violations (an increased late frontal positivity), and to event structure/animacy prediction violations (an increased late posterior positivity/P600). We argue that the late frontal positivity reflects a large change in activity associated with successfully updating the comprehender’s current situation model with new unpredicted information. We suggest that the late posterior positivity/P600 is triggered when the comprehender detects a conflict between the input and her model of the communicator and communicative environment. This leads to an initial failure to incorporate new unpredicted input, which may be followed by second-pass attempts to make sense of the discourse through reanalysis, repair, or reinterpretation. Together, these findings provide strong evidence that confirmed and violated predictions at different levels of representation manifest as distinct spatiotemporal neural signatures.

[1]  Ellen F. Lau,et al.  Wait a second! delayed impact of argument roles on on-line verb prediction , 2018 .

[2]  Colin M. Brown,et al.  The Mechanism Underlying Backward Priming in a Lexical Decision Task: Spreading Activation versus Semantic Matching , 1998 .

[3]  Ken McRae,et al.  People Use their Knowledge of Common Events to Understand Language, and Do So as Quickly as Possible , 2009, Lang. Linguistics Compass.

[4]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[5]  Michael J. Spivey,et al.  The Cambridge Handbook of Psycholinguistics , 2012 .

[6]  M. Kutas,et al.  Expect the Unexpected: Event-related Brain Response to Morphosyntactic Violations , 1998 .

[7]  M. Bar Predictions in the brain : using our past to generate a future , 2011 .

[8]  G. Kuperberg,et al.  Multiple Influences of Semantic Memory on Sentence Processing: Distinct Effects of Semantic Relatedness on Violations of Real-World Event/State Knowledge and Animacy Selection Restrictions. , 2012, Journal of memory and language.

[9]  Ron Sun,et al.  The Cambridge Handbook of Computational Psychology , 2008 .

[10]  Gina R. Kuperberg,et al.  Neural mechanisms of language comprehension: Challenges to syntax , 2007, Brain Research.

[11]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[12]  A. Borst Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[13]  C. Gallistel,et al.  The Perception of Probability , 2022 .

[14]  Gina R Kuperberg,et al.  What do we mean by prediction in language comprehension? , 2016, Language, cognition and neuroscience.

[15]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[16]  P. Schwanenflugel,et al.  Semantic relatedness and the scope of facilitation for upcoming words in sentences. , 1988 .

[17]  Edward W. Wlotko,et al.  Going the Extra Mile: Effects of Discourse Context on Two Late Positivities During Language Comprehension , 2020, Neurobiology of Language.

[18]  Mary Hare,et al.  Activating event knowledge , 2009, Cognition.

[19]  James L. McClelland Integrating probabilistic models of perception and interactive neural networks: a historical and tutorial review , 2013, Front. Psychol..

[20]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[21]  Jason Bohan,et al.  Anomalies at the Borderline of Awareness: An ERP Study , 2011, Journal of Cognitive Neuroscience.

[22]  Wendy Bergervoet,et al.  Monitoring in language perception , 2017 .

[23]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[24]  Dorothee J. Chwilla,et al.  Monitoring in Language Perception: Mild and Strong Conflicts Elicit Different ERP Patterns , 2010, Journal of Cognitive Neuroscience.

[25]  Rebecca Treiman,et al.  The English Lexicon Project , 2007, Behavior research methods.

[26]  Nelda Melissa Chavez Individual differences in verbal working memory, visuo-spatial working memory, and metacognition: Learning from text in a hypertext environment , 2002 .

[27]  Kerstin Preuschoff,et al.  Balancing New against Old Information: The Role of Puzzlement Surprise in Learning , 2018, Neural Computation.

[28]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[29]  J. Elman,et al.  Generalized event knowledge activation during online sentence comprehension. , 2012, Journal of memory and language.

[30]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[31]  P. Schumacher When combinatorial processing results in reconceptualization: toward a new approach of compositionality , 2013, Front. Psychol..

[32]  Kara D. Federmeier,et al.  A Beautiful Day in the Neighborhood: An Event-Related Potential Study of Lexical Relationships and Prediction in Context. , 2009, Journal of memory and language.

[33]  Gina R Kuperberg,et al.  Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events , 2016, Language, cognition and neuroscience.

[34]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[35]  P. Holcomb,et al.  Event-related brain potentials elicited by syntactic anomaly , 1992 .

[36]  Barbara H. Partee,et al.  Properties, types and meaning , 1988 .

[37]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[38]  M. Kutas In the company of other words: Electrophysiological evidence for single-word and sentence context effects , 1993 .

[39]  Gary S Dell,et al.  The P-chain: relating sentence production and its disorders to comprehension and acquisition , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[40]  Katherine A. DeLong,et al.  Predictability, plausibility, and two late ERP positivities during written sentence comprehension , 2014, Neuropsychologia.

[41]  T. Ditman,et al.  Electrophysiological insights into the processing of nominal metaphors , 2010, Neuropsychologia.

[42]  Phillip J. Holcomb,et al.  Neural correlates of processing syntactic, semantic, and thematic relationships in sentences , 2006 .

[43]  H. Schriefers,et al.  Prediction in language comprehension beyond specific words: An ERP study on sentence comprehension in Polish , 2013 .

[44]  Wilson L. Taylor,et al.  “Cloze Procedure”: A New Tool for Measuring Readability , 1953 .

[45]  H. H. Clark The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. , 1973 .

[46]  C. Van Petten,et al.  Lexical versus conceptual anticipation during sentence processing: frontal positivity and N400 ERP components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[47]  Jeffrey Gruber Studies in lexical relations , 1965 .

[48]  Richard N. Aslin,et al.  Incremental implicit learning of bundles of statistical patterns , 2016, Cognition.

[49]  Charles Kemp,et al.  Bayesian models of cognition , 2008 .

[50]  A. Friederici,et al.  Syntactic Gender and Semantic Expectancy: ERPs Reveal Early Autonomy and Late Interaction , 2000, Journal of Cognitive Neuroscience.

[51]  Aya Meltzer-Asscher,et al.  Lexical Inhibition due to Failed Prediction: Behavioral Evidence and ERP Correlates , 2017, Journal of experimental psychology. Learning, memory, and cognition.

[52]  Ellen F. Lau,et al.  A Direct Comparison of N400 Effects of Predictability and Incongruity in Adjective-Noun Combination , 2016 .

[53]  M. Traxler,et al.  Effects of prediction and contextual support on lexical processing: Prediction takes precedence , 2015, Cognition.

[54]  Amy Beth Warriner,et al.  Concreteness ratings for 40 thousand generally known English word lemmas , 2014, Behavior research methods.

[55]  L. Osterhout,et al.  The independence of combinatory semantic processing: Evidence from event-related potentials , 2005 .

[56]  Laurence Aitchison,et al.  With or without you: predictive coding and Bayesian inference in the brain , 2017, Current Opinion in Neurobiology.

[57]  Colin M. Brown,et al.  The syntactic positive shift (sps) as an erp measure of syntactic processing , 1993 .

[58]  J. Nicol,et al.  On the Distinctiveness, Independence, and Time Course of the Brain Responses to Syntactic and Semantic Anomalies. , 1999 .

[59]  Edward J. Alexander,et al.  1 Supplementary Material : What we Know about Knowing : Presuppositions generated by factive verbs influence downstream neural processing , 2018 .

[60]  Marc Brysbaert,et al.  Moving beyond Kučera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English , 2009, Behavior research methods.

[61]  Jelena Mirkovic,et al.  Incrementality and Prediction in Human Sentence Processing , 2009, Cogn. Sci..

[62]  Pierre Baldi,et al.  Of bits and wows: A Bayesian theory of surprise with applications to attention , 2010, Neural Networks.

[63]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[64]  Marta Kutas,et al.  CHAPTER 15 A Look around at What Lies Ahead: Prediction and Predictability in Language Processing , 2010 .

[65]  Ellen F. Lau,et al.  Dissociating N400 Effects of Prediction from Association in Single-word Contexts , 2013, Journal of Cognitive Neuroscience.

[66]  Judith F. Kroll,et al.  Cognitive control ability mediates prediction costs in monolinguals and bilinguals , 2018, Cognition.

[67]  Martin Paczynski,et al.  Establishing Causal Coherence across Sentences: An ERP Study , 2011, Journal of Cognitive Neuroscience.

[68]  M. Kutas,et al.  Anticipating Words and Their Gender: An Event-related Brain Potential Study of Semantic Integration, Gender Expectancy, and Gender Agreement in Spanish Sentence Reading , 2004, Journal of Cognitive Neuroscience.

[69]  Laurie E. Cutting,et al.  Unraveling reading comprehension : behavioral, neurobiological, and genetic components , 2013 .

[70]  Gina R Kuperberg,et al.  Electrophysiological evidence for use of the animacy hierarchy, but not thematic role assignment, during verb-argument processing , 2011, Language and cognitive processes.

[71]  Steven J. Luck,et al.  ERPLAB: an open-source toolbox for the analysis of event-related potentials , 2014, Front. Hum. Neurosci..

[72]  Kara D. Federmeier,et al.  Age-related and individual differences in the use of prediction during language comprehension , 2010, Brain and Language.

[73]  Hartmut Fitz,et al.  Getting real about Semantic Illusions: Rethinking the functional role of the P600 in language comprehension , 2012, Brain Research.

[74]  Colin M. Brown,et al.  When and how do listeners relate a sentence to the wider discourse? Evidence from the N400 effect. , 2003, Brain research. Cognitive brain research.

[75]  D. Caplan,et al.  The role of animacy and thematic relationships in processing active English sentences: Evidence from event-related potentials , 2007, Brain and Language.

[76]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[77]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[78]  James L. McClelland,et al.  Modelling the N400 brain potential as change in a probabilistic representation of meaning , 2018, Nature Human Behaviour.

[79]  Christian J. Fiebach,et al.  Finding the P3 in the P600: Decoding shared neural mechanisms of responses to syntactic violations and oddball targets , 2019, NeuroImage.

[80]  G. Kuperberg,et al.  Reversing expectations during discourse comprehension , 2015, Language, cognition and neuroscience.

[81]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[82]  Hiroko Nakano,et al.  Speech and Span: Working Memory Capacity Impacts the Use of Animacy but Not of World Knowledge during Spoken Sentence Comprehension , 2010, Journal of Cognitive Neuroscience.

[83]  Patrick F. Reidy An Introduction to Latent Semantic Analysis , 2009 .

[84]  Rajesh P. N. Rao,et al.  Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex , 1997, Neural Computation.

[85]  R. Jackendoff The Status of Thematic Relations in Linguistic Theory , 1987 .

[86]  N. Chater,et al.  Rational models of cognition , 1998 .

[87]  Peter,et al.  Semantic Unification , 2008 .

[88]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[89]  Mante S. Nieuwland,et al.  When Peanuts Fall in Love: N400 Evidence for the Power of Discourse , 2005, Journal of Cognitive Neuroscience.

[90]  Jerome L. Myers,et al.  Accessing the discourse representation during reading , 1998 .

[91]  Dave F. Kleinschmidt,et al.  Robust speech perception: recognize the familiar, generalize to the similar, and adapt to the novel. , 2015, Psychological review.

[92]  Richard N. Aslin,et al.  Learning to Represent a Multi-Context Environment: More than Detecting Changes , 2012, Front. Psychology.

[93]  Noah D. Goodman,et al.  Wonky worlds: Listeners revise world knowledge when utterances are odd , 2015, CogSci.

[94]  Kara D. Federmeier Thinking ahead: the role and roots of prediction in language comprehension. , 2007, Psychophysiology.

[95]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[96]  C. Van Petten,et al.  Prediction during language comprehension: benefits, costs, and ERP components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[97]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[98]  Michael W. Spratling Predictive coding as a model of biased competition in visual attention , 2008, Vision Research.

[99]  David R. Dowty On the Semantic Content of the Notion of ‘Thematic Role’ , 1989 .

[100]  Andrea Weber,et al.  When One Person's Mistake Is Another's Standard Usage: The Effect of Foreign Accent on Syntactic Processing , 2012, Journal of Cognitive Neuroscience.

[101]  Heather J. Ferguson,et al.  Dissociable effects of prediction and integration during language comprehension: Evidence from a large-scale study using brain potentials , 2018, bioRxiv.

[102]  E. Bizzi,et al.  The Cognitive Neurosciences , 1996 .

[103]  J. Toomasian The Case for the Case , 2016, Perfusion.

[104]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[105]  D. Caplan,et al.  Electrophysiological distinctions in processing conceptual relationships within simple sentences. , 2003, Brain research. Cognitive brain research.

[106]  Michael C. Frank,et al.  Predicting Pragmatic Reasoning in Language Games , 2012, Science.

[107]  Kara D. Federmeier,et al.  Multiple effects of sentential constraint on word processing , 2007, Brain Research.

[108]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[109]  Floris P. de Lange,et al.  How Prediction Errors Shape Perception, Attention, and Motivation , 2012, Front. Psychology.

[110]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[111]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.

[112]  Pienie Zwitserlood,et al.  Dissociating predictability, plausibility and possibility of sentence continuations in reading: evidence from late-positivity ERPs , 2018, PeerJ.

[113]  Colin Phillips,et al.  A “bag-of-arguments” mechanism for initial verb predictions , 2016 .

[114]  G. Dell,et al.  Becoming syntactic. , 2006, Psychological review.

[115]  David Mumford,et al.  On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[116]  Dietmar Roehm,et al.  Semantic prediction in language comprehension: evidence from brain potentials , 2016, Language, cognition and neuroscience.

[117]  M. Schlesewsky,et al.  The P600-as-P3 hypothesis revisited: Single-trial analyses reveal that the late EEG positivity following linguistically deviant material is reaction time aligned , 2014, Brain and Language.

[118]  D Mumford,et al.  On the computational architecture of the neocortex. II. The role of cortico-cortical loops. , 1992, Biological cybernetics.

[119]  H. Kolk,et al.  Structure and limited capacity in verbal working memory: A study with event-related potentials , 2003, Brain and Language.

[120]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[121]  David M. Groppe,et al.  Overlapping dual ERP responses to low cloze probability sentence continuations. , 2011, Psychophysiology.