Modelling the N400 brain potential as change in a probabilistic representation of meaning

The N400 component of the event-related brain potential has aroused much interest because it is thought to provide an online measure of meaning processing in the brain. However, the underlying process remains incompletely understood and actively debated. Here we present a computationally explicit account of this process and the emerging representation of sentence meaning. We simulate N400 amplitudes as the change induced by an incoming stimulus in an implicit and probabilistic representation of meaning captured by the hidden unit activation pattern in a neural network model of sentence comprehension, and we propose that the process underlying the N400 also drives implicit learning in the network. The model provides a unified account of 16 distinct findings from the N400 literature and connects human language comprehension with recent deep learning approaches to language processing.The N400 evoked potential is a window to meaning in the brain, but it remains incompletely understood. The authors provide a unified explanation of the N400 in a neural network model that avoids the commitments of traditional approaches to meaning in language.

[1]  D. Caplan,et al.  Electrophysiological distinctions in processing conceptual relationships within simple sentences. , 2003, Brain research. Cognitive brain research.

[2]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[3]  Kara D. Federmeier,et al.  Multiple effects of sentential constraint on word processing , 2007, Brain Research.

[4]  Peter Hagoort,et al.  The Processing Nature of the N400: Evidence from Masked Priming , 1993, Journal of Cognitive Neuroscience.

[5]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[6]  E. Vogel,et al.  Word meanings can be accessed but not reported during the attentional blink , 1996, Nature.

[7]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.

[8]  A. Sanford,et al.  Depth of processing in language comprehension: not noticing the evidence , 2002, Trends in Cognitive Sciences.

[9]  Janet L. McDonald,et al.  The Development of Sentence Comprehension Strategies in English and Dutch. , 1986 .

[10]  M. Koivisto,et al.  Cognitive representations underlying the N400 priming effect. , 2001, Brain research. Cognitive brain research.

[11]  Colin M. Brown,et al.  The cascaded nature of lexical selection and integration in auditory sentence processing. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[12]  M. Kutas,et al.  Influences of semantic and syntactic context on open- and closed-class words , 1991, Memory & cognition.

[13]  E. Plante,et al.  Time course of word identification and semantic integration in spoken language. , 1999, Journal of experimental psychology. Learning, memory, and cognition.

[14]  A. Borst Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[15]  Mante S. Nieuwland,et al.  When the Truth Is Not Too Hard to Handle , 2008, Psychological science.

[16]  Colin M. Brown,et al.  Semantic Integration in Sentences and Discourse: Evidence from the N400 , 1999, Journal of Cognitive Neuroscience.

[17]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[18]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[19]  M. Rugg The effects of semantic priming and work repetition on event-related potentials. , 1985, Psychophysiology.

[20]  H. Barber,et al.  Syllable-frequency effects in visual word recognition: evidence from ERPs , 2004, Neuroreport.

[21]  P. Holcomb,et al.  Event-related brain potentials elicited by syntactic anomaly , 1992 .

[22]  D. Plaut,et al.  A neurally plausible Parallel Distributed Processing model of Event-Related Potential word reading data , 2012, Brain and Language.

[23]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[24]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[25]  Geoffrey E. Hinton Learning and Applying Contextual Constraints in Sentence Comprehension , 1991 .

[26]  Giosuè Baggio,et al.  The balance between memory and unification in semantics: A dynamic account of the N400 , 2011 .

[27]  H. Heinze,et al.  Perceptual Priming Versus Explicit Memory: Dissociable Neural Correlates at Encoding , 2002, Journal of Cognitive Neuroscience.

[28]  J. Debruille,et al.  The N400 potential could index a semantic inhibition , 2007, Brain Research Reviews.

[29]  Salim Roukos,et al.  Brain potentials related to stages of sentence verification. , 1983, Psychophysiology.

[30]  M. Kutas,et al.  An Event-Related Potential (ERP) Analysis of Semantic Congruity and Repetition Effects in Sentences , 1992, Journal of Cognitive Neuroscience.

[31]  James L. McClelland,et al.  The parallel distributed processing approach to semantic cognition , 2003, Nature Reviews Neuroscience.

[32]  Ann Marie Schmidt,et al.  The Receptor for Advanced Glycation End Products (RAGE) Affects T Cell Differentiation in OVA Induced Asthma , 2014, PloS one.

[33]  Matthew W. Crocker,et al.  A Neurocomputational Model of the N400 and the P600 in Language Processing , 2016, Cognitive science.

[34]  H. Kolk,et al.  An ERP study of P600 effects elicited by semantic anomalies. , 2005, Brain research. Cognitive brain research.

[35]  Brian MacWhinney,et al.  The handbook of language emergence , 2015 .

[36]  G. Altmann,et al.  Incremental interpretation at verbs: restricting the domain of subsequent reference , 1999, Cognition.

[37]  L. Osterhout,et al.  Neural correlates of second-language word learning: minimal instruction produces rapid change , 2004, Nature Neuroscience.

[38]  Colin M. Brown,et al.  ERP effects of listening to speech compared to reading: the P600/SPS to syntactic violations in spoken sentences and rapid serial visual presentation , 2000, Neuropsychologia.

[39]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[40]  Kara D. Federmeier,et al.  Chapter 1 Time for Meaning: Electrophysiology Provides Insights into the Dynamics of Representation and Processing in Semantic Memory , 2009 .

[41]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[42]  Yvonne Koch The Psychology Of Learning And Motivation , 2016 .

[43]  Brian A. Wandell,et al.  THE COGNITIVE NEUROSCIENCES Fourth Edition , 2009 .

[44]  M. Farah,et al.  Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[45]  M. Kutas,et al.  Expect the Unexpected: Event-related Brain Response to Morphosyntactic Violations , 1998 .

[46]  John C J Hoeks,et al.  Seeing words in context: the interaction of lexical and sentence level information during reading. , 2004, Brain research. Cognitive brain research.

[47]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[48]  J. Polich Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[49]  Geoffrey E. Hinton Connectionist Learning Procedures , 1989, Artif. Intell..

[50]  Roger Levy,et al.  A noisy-channel model of rational human sentence comprehension under uncertain input , 2008, EMNLP 2008.

[51]  A. Turken,et al.  The Neural Architecture of the Language Comprehension Network: Converging Evidence from Lesion and Connectivity Analyses , 2011, Front. Syst. Neurosci..

[52]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[53]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[54]  Angela D. Friederici,et al.  Isn't It Ironic? An Electrophysiological Exploration of Figurative Language Processing , 2011, Journal of Cognitive Neuroscience.

[55]  Gina R Kuperberg,et al.  Separate streams or probabilistic inference? What the N400 can tell us about the comprehension of events , 2016, Language, cognition and neuroscience.

[56]  Milena Rabovsky,et al.  Simulating the N400 ERP component as semantic network error: Insights from a feature-based connectionist attractor model of word meaning , 2014, Cognition.

[57]  Gabriella Vigliocco,et al.  Integrating experiential and distributional data to learn semantic representations. , 2009, Psychological review.

[58]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[59]  W. Sommer,et al.  Differential Task Effects on N400 and P600 Elicited by Semantic and Syntactic Violations , 2014, PloS one.

[60]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[61]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[62]  Karl G. D. Bailey,et al.  Good-Enough Representations in Language Comprehension , 2002 .

[63]  Lukás Burget,et al.  Strategies for training large scale neural network language models , 2011, 2011 IEEE Workshop on Automatic Speech Recognition & Understanding.

[64]  R. Jacobs,et al.  Perception of speech reflects optimal use of probabilistic speech cues , 2008, Cognition.

[65]  A. Friederici,et al.  N400-like Semantic Incongruity Effect in 19-Month-Olds: Processing Known Words in Picture Contexts , 2004, Journal of Cognitive Neuroscience.

[66]  S. Bookheimer Functional MRI of language: new approaches to understanding the cortical organization of semantic processing. , 2002, Annual review of neuroscience.

[67]  Blair C. Armstrong,et al.  PSPs and ERPs: Applying the dynamics of post-synaptic potentials to individual units in simulation of temporally extended Event-Related Potential reading data , 2014, Brain and Language.

[68]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[69]  A. Jongman,et al.  A comparison of semantic and syntactic event related potentials generated by children and adults , 2006, Brain and Language.

[70]  Mante S. Nieuwland,et al.  When Peanuts Fall in Love: N400 Evidence for the Power of Discourse , 2005, Journal of Cognitive Neuroscience.

[71]  S. Gotts Incremental learning of perceptual and conceptual representations and the puzzle of neural repetition suppression , 2016, Psychonomic bulletin & review.

[72]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[73]  Bruce D. McCandliss,et al.  The visual word form area: expertise for reading in the fusiform gyrus , 2003, Trends in Cognitive Sciences.

[74]  Samuel J Cheyette,et al.  Modeling the N400 ERP component as transient semantic over-activation within a neural network model of word comprehension , 2017, Cognition.

[75]  G. McCarthy,et al.  Language-related field potentials in the anterior-medial temporal lobe: II. Effects of word type and semantic priming , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[76]  G. McCarthy,et al.  Language-related field potentials in the anterior-medial temporal lobe: I. Intracranial distribution and neural generators , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[77]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[78]  L. Osterhout,et al.  The independence of combinatory semantic processing: Evidence from event-related potentials , 2005 .

[79]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[80]  N. Dronkers,et al.  Lesion analysis of the brain areas involved in language comprehension , 2004, Cognition.

[81]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[82]  M. Schlesewsky,et al.  The P600-as-P3 hypothesis revisited: Single-trial analyses reveal that the late EEG positivity following linguistically deviant material is reaction time aligned , 2014, Brain and Language.

[83]  S. Frank,et al.  The ERP response to the amount of information conveyed by words in sentences , 2015, Brain and Language.

[84]  Katherine A. DeLong,et al.  Predictability, plausibility, and two late ERP positivities during written sentence comprehension , 2014, Neuropsychologia.

[85]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[86]  R. Jackendoff Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002 .

[87]  Stephen Budiansky Green house effect: Energy efficiency may help , 1984, Nature.

[88]  M. Kutas,et al.  The N400 in a semantic categorization task across 6 decades. , 1998, Electroencephalography and clinical neurophysiology.