How well does surprisal explain N400 amplitude under different experimental conditions?

We investigate the extent to which word surprisal can be used to predict a neural measure of human language processing difficulty - the N400. To do this, we use recurrent neural networks to calculate the surprisal of stimuli from previously published neurolinguistic studies of the N400. We find that surprisal can predict N400 amplitude in a wide range of cases, and the cases where it cannot do so provide valuable insight into the neurocognitive processes underlying the response.

[1]  Milena Rabovsky,et al.  Simulating the N400 ERP component as semantic network error: Insights from a feature-based connectionist attractor model of word meaning , 2014, Cognition.

[2]  Roger Levy,et al.  What do RNN Language Models Learn about Filler–Gap Dependencies? , 2018, BlackboxNLP@EMNLP.

[3]  Roger Levy,et al.  Neural language models as psycholinguistic subjects: Representations of syntactic state , 2019, NAACL.

[4]  I. Fischler,et al.  Completion norms for 329 sentence contexts , 1980, Memory & cognition.

[5]  M. Kutas.  In the company of other words: Electrophysiological evidence for single-word and sentence context effects , 1993 .

[6]  Marta Kutas,et al.  Quantifiers more or less quantify online: ERP evidence for partial incremental interpretation. , 2010, Journal of memory and language.

[7]  Roger Levy,et al.  Structural Supervision Improves Learning of Non-Local Grammatical Dependencies , 2019, NAACL.

[8]  Edouard Grave,et al.  Colorless Green Recurrent Networks Dream Hierarchically , 2018, NAACL.

[9]  C. J. McGrath,et al.  Effect of exchange rate return on volatility spill-over across trading regions , 2012 .

[10]  Matthew W. Crocker,et al.  Expectation-based Comprehension: Modeling the Interaction of World Knowledge and Linguistic Experience , 2019 .

[11]  C. Van Petten,et al.  Prediction during language comprehension: benefits, costs, and ERP components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[12]  Antal van den Bosch,et al.  Prediction During Natural Language Comprehension. , 2016, Cerebral cortex.

[13]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[14]  L. Osterhout,et al.  The independence of combinatory semantic processing: Evidence from event-related potentials , 2005 .

[15]  Gabriella Vigliocco,et al.  Lexical surprisal as a general predictor of reading time , 2012, EACL.

[16]  Blair C. Armstrong,et al.  PSPs and ERPs: Applying the dynamics of post-synaptic potentials to individual units in simulation of temporally extended Event-Related Potential reading data , 2014, Brain and Language.

[17]  James L. McClelland,et al.  Modelling the N400 brain potential as change in a probabilistic representation of meaning , 2018, Nature Human Behaviour.

[18]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[19]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[20]  No Value,et al.  Proceedings of the 25th annual conference of the cognitive science society , 2001 .

[21]  S. Frank,et al.  The ERP response to the amount of information conveyed by words in sentences , 2015, Brain and Language.

[22]  Brian Roark,et al.  Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing , 2009, EMNLP.

[23]  Matthew W. Crocker,et al.  A Neurocomputational Model of the N400 and the P600 in Language Processing , 2016, Cognitive science.

[24]  Roel M. Willems,et al.  Word predictability and semantic similarity show distinct patterns of brain activity during language comprehension , 2017 .

[25]  Allyson Ettinger,et al.  Modeling N400 amplitude using vector space models of word representation , 2016, CogSci.

[26]  Wilson L. Taylor,et al.  “Cloze Procedure”: A New Tool for Measuring Readability , 1953 .

[27]  Stefan Frank,et al.  Modelling Reading Times in Bilingual Sentence Comprehension , 2014, CogSci.

[28]  Thorsten Brants,et al.  One billion word benchmark for measuring progress in statistical language modeling , 2013, INTERSPEECH.

[29]  John Hoeks,et al.  Modeling the Noun Phrase versus Sentence Coordination Ambiguity in Dutch: Evidence from Surprisal Theory , 2010, CMCL@ACL.

[30]  Rui P. Chaves,et al.  Assessing the ability of Transformer-based Neural Models to represent structurally unbounded dependencies , 2020, SCIL.

[31]  Yonghui Wu,et al.  Exploring the Limits of Language Modeling , 2016, ArXiv.

[32]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[33]  Franklin Chang,et al.  Language ERPs reflect learning through prediction error propagation , 2019, Cognitive Psychology.

[34]  Stefan Frank,et al.  Early effects of word surprisal on pupil size during reading , 2012, CogSci.

[35]  Frank Keller,et al.  Cognitively Plausible Models of Human Language Processing , 2010, ACL.

[36]  L. Osterhout,et al.  Event-Related Brain Potentials Elicited by Failure to Agree , 1995 .

[37]  Nathaniel J. Smith,et al.  The effect of word predictability on reading time is logarithmic , 2013, Cognition.

[38]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[39]  Steven G. Luke,et al.  Limits on lexical prediction during reading , 2016, Cognitive Psychology.

[40]  Gina R Kuperberg,et al.  Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events , 2016, Language, cognition and neuroscience.

[41]  Peng Qian,et al.  Representation of Constituents in Neural Language Models: Coordination Phrase as a Case Study , 2019, EMNLP.

[42]  Gina R. Kuperberg,et al.  A Tale of Two Positivities (and the N400): Distinct neural signatures are evoked by confirmed and violated predictions at different levels of representation , 2018, bioRxiv.

[43]  Katherine A. DeLong,et al.  Comprehending surprising sentences: sensitivity of post-N400 positivities to contextual congruity and semantic relatedness , 2020, Language, cognition and neuroscience.

[44]  Frank Keller,et al.  Syntactic and Semantic Factors in Processing Difficulty: An Integrated Measure , 2010, ACL.

[45]  Kim Ainsworth-Darnell,et al.  Dissociating Brain Responses to Syntactic and Semantic Anomalies: Evidence from Event-Related Potentials , 1998 .

[46]  Mark Johnson,et al.  Using Language Models and Latent Semantic Analysis to Characterise the N400m Neural Response , 2011, ALTA.

[47]  Stefan L. Frank,et al.  Evaluating information-theoretic measures of word prediction in naturalistic sentence reading , 2019, Neuropsychologia.

[48]  R. Levy.  Expectation-based syntactic comprehension , 2008, Cognition.

[49]  Per B. Brockhoff,et al.  lmerTest Package: Tests in Linear Mixed Effects Models , 2017 .

[50]  D. Plaut,et al.  A neurally plausible Parallel Distributed Processing model of Event-Related Potential word reading data , 2012, Brain and Language.

[51]  Mante S. Nieuwland,et al.  Predicting form and meaning: Evidence from brain potentials , 2016 .

[52]  Ellen F. Lau,et al.  Comprehenders Rationally Adapt Semantic Predictions to the Statistics of the Local Environment: a Bayesian Model of Trial-by-Trial N400 Amplitudes , 2017, CogSci.

[53]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[54]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[55]  Roger Levy,et al.  Sequential vs. Hierarchical Syntactic Models of Human Incremental Sentence Processing , 2012, CMCL@NAACL-HLT.

[56]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[57]  Samuel J Cheyette,et al.  Modeling the N400 ERP component as transient semantic over-activation within a neural network model of word comprehension , 2017, Cognition.

[58]  Reinhold Kliegl,et al.  Parsing costs as predictors of reading difficulty: An evaluation using the Potsdam Sentence Corpus , 2008, Journal of Eye Movement Research.

[59]  Nathaniel J. Smith,et al.  Optimal Processing Times in Reading: A Formal Model and Empirical Investigation , 2008 .

[60]  Frank Keller,et al.  Data from eye-tracking corpora as evidence for theories of syntactic processing complexity , 2008, Cognition.