Delta-band Cortical Tracking of Acoustic and Linguistic Features in Natural Spoken Narratives

Speech contains rich acoustic and linguistic information. During speech comprehension, cortical activity tracks the acoustic envelope of speech. Recent studies also observe cortical tracking of higher-level linguistic units, such as words and phrases, using synthesized speech deprived of delta-band acoustic envelope. It remains unclear, however, how cortical activity jointly encodes the acoustic and linguistic information in natural speech. Here, we investigate the neural encoding of words and demonstrate that delta-band cortical activity tracks the rhythm of multi-syllabic words when naturally listening to narratives. Furthermore, by dissociating the word rhythm from acoustic envelope, we find cortical activity primarily tracks the word rhythm during speech comprehension. When listeners’ attention is diverted, however, neural tracking of words diminishes, and delta-band activity becomes phase locked to the acoustic envelope. These results suggest that large-scale cortical dynamics in the delta band are primarily coupled to the rhythm of linguistic units during natural speech comprehension.

[1]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[2]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[3]  Lars Meyer,et al.  Synchronization of Electrophysiological Responses with Speech Benefits Syntactic Information Processing , 2018, Journal of Cognitive Neuroscience.

[4]  Antoine J. Shahin,et al.  Attentional Gain Control of Ongoing Cortical Speech Representations in a “Cocktail Party” , 2010, The Journal of Neuroscience.

[5]  Joachim Gross,et al.  Phase-Locked Responses to Speech in Human Auditory Cortex are Enhanced During Comprehension , 2012, Cerebral cortex.

[6]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[7]  David Poeppel,et al.  Discrimination of speech stimuli based on neuronal response phase patterns depends on acoustics but not comprehension. , 2010, Journal of neurophysiology.

[8]  San Duanmu Stress in Chinese , 2001 .

[9]  Christopher K. Kovach,et al.  Temporal Envelope of Time-Compressed Speech Represented in the Human Auditory Cortex , 2009, The Journal of Neuroscience.

[10]  Edmund C. Lalor,et al.  Low-Frequency Cortical Entrainment to Speech Reflects Phoneme-Level Processing , 2015, Current Biology.

[11]  L. Pylkkänen,et al.  Neuromagnetic Evidence for the Timing of Lexical Activation: An MEG Component Sensitive to Phonotactic Probability but Not to Neighborhood Density , 2002, Brain and Language.

[12]  John J. Foxe,et al.  Attentional Selection in a Cocktail Party Environment Can Be Decoded from Single-Trial EEG. , 2015, Cerebral cortex.

[13]  J. Simon,et al.  Emergence of neural encoding of auditory objects while listening to competing speakers , 2012, Proceedings of the National Academy of Sciences.

[14]  L. Elliot Hong,et al.  Rapid Transformation from Auditory to Linguistic Representations of Continuous Speech , 2018, Current Biology.

[15]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[16]  Joachim Gross,et al.  Simple Acoustic Features Can Explain Phoneme-Based Predictions of Cortical Responses to Speech , 2019, Current Biology.

[17]  Benedikt Zoefel,et al.  EEG oscillations entrain their phase to high-level features of speech sound , 2016, NeuroImage.

[18]  James W. Minett,et al.  Delta, theta, beta, and gamma brain oscillations index levels of auditory sentence processing , 2016, NeuroImage.

[19]  D. Poeppel,et al.  Sensitivity to temporal modulation rate and spectral bandwidth in the human auditory system: fMRI evidence. , 2012, Journal of neurophysiology.

[20]  David Poeppel,et al.  Sensitivity to temporal modulation rate and spectral bandwidth in the human auditory system: MEG evidence. , 2012, Journal of neurophysiology.

[21]  Nai Ding,et al.  Eye activity tracks task-relevant structures during speech and auditory sequence perception , 2018, Nature Communications.

[22]  Ramesh Srinivasan,et al.  Suppression of competing speech through entrainment of cortical oscillations. , 2013, Journal of neurophysiology.

[23]  S. David,et al.  Encoding of natural sounds by variance of the cortical local field potential. , 2016, Journal of neurophysiology.

[24]  Jingwei Sheng,et al.  The Cortical Maps of Hierarchical Linguistic Structures during Speech Perception. , 2018, Cerebral cortex.

[25]  Mathieu Bourguignon,et al.  Comparing the potential of MEG and EEG to uncover brain tracking of speech temporal envelope , 2019, NeuroImage.

[26]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[27]  Jing Zheng,et al.  Auditory and language contributions to neural encoding of speech features in noisy environments , 2018, NeuroImage.

[28]  Cheng Luo,et al.  Auditory and Language Contributions to Neural Encoding of Speech Features in Noisy Environments , 2018 .

[29]  S. Shamma,et al.  Temporal coherence and attention in auditory scene analysis , 2011, Trends in Neurosciences.

[30]  Virginie van Wassenhove,et al.  Distinct contributions of low- and high-frequency neural oscillations to speech comprehension , 2017 .

[31]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[32]  Wen Zhang,et al.  Attention Is Required for Knowledge-Based Sequential Grouping: Insights from the Integration of Syllables into Words , 2017, The Journal of Neuroscience.

[33]  Mika Seppä,et al.  Uncovering cortical MEG responses to listened audiobook stories , 2014, NeuroImage.

[34]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[35]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[36]  Usha Goswami,et al.  Speech rhythm and language acquisition: an amplitude modulation phase hierarchy perspective , 2019, Annals of the New York Academy of Sciences.

[37]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[38]  Jan Wouters,et al.  Speech Intelligibility Predicted from Neural Entrainment of the Speech Envelope , 2018, bioRxiv.

[39]  David Poeppel,et al.  Speech rhythms and their neural foundations , 2020, Nature Reviews Neuroscience.

[40]  David Poeppel,et al.  Acoustic landmarks drive delta–theta oscillations to enable speech comprehension by facilitating perceptual parsing , 2014, NeuroImage.

[41]  Y. Nir,et al.  Sleep Disrupts High-Level Speech Parsing Despite Significant Basic Auditory Processing , 2017, The Journal of Neuroscience.

[42]  B. Shinn-Cunningham Object-based auditory and visual attention , 2008, Trends in Cognitive Sciences.

[43]  Ying-Yee Kong,et al.  Differential modulation of auditory responses to attended and unattended speech in different listening conditions , 2014, Hearing Research.

[44]  L. Pylkkänen,et al.  Tracking the time course of word recognition with MEG , 2003, Trends in Cognitive Sciences.

[45]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[46]  D. Poeppel,et al.  Cortical Tracking of Hierarchical Linguistic Structures in Connected Speech , 2015, Nature Neuroscience.

[47]  Marco Buiatti,et al.  Investigating the neural correlates of continuous speech computation with frequency-tagged neuroelectric responses , 2009, NeuroImage.

[48]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[49]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[50]  Lucia Melloni,et al.  Brain Oscillations during Spoken Sentence Processing , 2012, Journal of Cognitive Neuroscience.

[51]  Tobias Reichenbach,et al.  Neural Speech Tracking in the Theta and in the Delta Frequency Band Differentially Encode Clarity and Comprehension of Speech in Noise , 2019, The Journal of Neuroscience.

[52]  E. Erdfelder,et al.  Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses , 2009, Behavior research methods.

[53]  Ellen F. Lau,et al.  A cortical network for semantics: (de)constructing the N400 , 2008, Nature Reviews Neuroscience.

[54]  John J. Foxe,et al.  Resolving precise temporal processing properties of the auditory system using continuous stimuli. , 2009, Journal of neurophysiology.

[55]  Andrea E. Martin,et al.  Abstraction in time: Finding hierarchical linguistic structure in a model of relational processing , 2016, CogSci.

[56]  Joachim Gross,et al.  Perceptually relevant speech tracking in auditory and motor cortex reflects distinct linguistic features , 2018, PLoS biology.

[57]  A. Friederici The cortical language circuit: from auditory perception to sentence comprehension , 2012, Trends in Cognitive Sciences.

[58]  Michael J. Crosse,et al.  Electrophysiological correlates of semantic dissimilarity reflect the comprehension of natural, narrative speech , 2017, bioRxiv.

[59]  Lisa D. Sanders,et al.  An ERP study of continuous speech processing. I. Segmentation, semantics, and syntax in native speakers. , 2003, Brain research. Cognitive brain research.