Retrieving Event-related Human Brain Dynamics from Natural Sentence Reading

Electroencephalography (EEG) signals recordings when people reading natural languages are commonly used as a cognitive method to interpret human language understanding in neuroscience and psycholinguistics. Previous studies have demonstrated that the human fixation and activation in word reading associated with some brain regions, but it is not clear when and how to measure the brain dynamics across time and frequency domains. In this study, we propose the first analysis of event-related brain potentials (ERPs), and event-related spectral perturbations (ERSPs) on benchmark datasets which consist of sentence-level simultaneous EEG and related eye-tracking recorded from human natural reading experiment tasks. Our results showed peaks evoked at around 162 ms after the stimulus (starting to read each sentence) in the occipital area, indicating the brain retriving lexical and semantic visual information processing approaching 200 ms from the sentence onset. Furthermore, the occipital ERP around 200ms presents negative power and positive power in short and long reaction times. In addition, the occipital ERSP around 200ms demonstrated increased high gamma and decreased low beta and low gamma power, relative to the baseline. Our results implied that most of the semantic-perception responses occurred around the 200ms in alpha, beta and gamma bands of EEG signals. Our findings also provide potential impacts on promoting cognitive natural language processing models evaluation from EEG dynamics.

[1]  Nora Hollenstein,et al.  ZuCo, a simultaneous EEG and eye-tracking resource for natural sentence reading , 2018, Scientific Data.

[2]  M. Just,et al.  Neuroimaging Contributions to the Understanding of Discourse Processes , 2006 .

[3]  P. Leppänen,et al.  Semantic anomaly detection in school-aged children during natural sentence reading – A study of fixation-related brain potentials , 2018, PloS one.

[4]  S. Frank,et al.  The ERP response to the amount of information conveyed by words in sentences , 2015, Brain and Language.

[5]  A. Jacobs,et al.  Coregistration of eye movements and EEG in natural reading: analyses and review. , 2011, Journal of experimental psychology. General.

[6]  Y. Chen,et al.  The time-course of single-word reading: Evidence from fast behavioral and brain responses , 2012, NeuroImage.

[7]  Tom Mitchell,et al.  Relating Simple Sentence Representations in Deep Neural Networks and the Brain , 2019, ACL.

[8]  Robin L Thompson,et al.  Reading time data for evaluating broad-coverage models of English sentence processing , 2013, Behavior research methods.

[9]  John Hale,et al.  Finding syntax in human encephalography with beam search , 2018, ACL.

[10]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[11]  D. Mathalon,et al.  Event-related EEG time-frequency analysis: an overview of measures and an analysis of early gamma band phase locking in schizophrenia. , 2008, Schizophrenia bulletin.

[12]  I. Peretz,et al.  Musical scale properties are automatically processed in the human auditory cortex , 2006, Brain Research.

[13]  Nora Hollenstein,et al.  ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation , 2020, LREC.