Tracking cortical entrainment in neural activity: auditory processes in human temporal cortex

A primary objective for cognitive neuroscience is to identify how features of the sensory environment are encoded in neural activity. Current auditory models of loudness perception can be used to make detailed predictions about the neural activity of the cortex as an individual listens to speech. We used two such models (loudness-sones and loudness-phons), varying in their psychophysiological realism, to predict the instantaneous loudness contours produced by 480 isolated words. These two sets of 480 contours were used to search for electrophysiological evidence of loudness processing in whole-brain recordings of electro- and magneto-encephalographic (EMEG) activity, recorded while subjects listened to the words. The technique identified a bilateral sequence of loudness processes, predicted by the more realistic loudness-sones model, that begin in auditory cortex at ~80 ms and subsequently reappear, tracking progressively down the superior temporal sulcus (STS) at lags from 230 to 330 ms. The technique was then extended to search for regions sensitive to the fundamental frequency (F0) of the voiced parts of the speech. It identified a bilateral F0 process in auditory cortex at a lag of ~90 ms, which was not followed by activity in STS. The results suggest that loudness information is being used to guide the analysis of the speech stream as it proceeds beyond auditory cortex down STS toward the temporal pole.

[1]  S. David,et al.  Influence of context and behavior on stimulus reconstruction from neural activity in primary auditory cortex. , 2009, Journal of neurophysiology.

[2]  J. Obleser,et al.  Entrained neural oscillations in multiple frequency bands comodulate behavior , 2014, Proceedings of the National Academy of Sciences.

[3]  Steven Greenberg,et al.  On the Possible Role of Brain Rhythms in Speech Perception: Intelligibility of Time-Compressed Speech with Periodic and Aperiodic Insertions of Silence , 2009, Phonetica.

[4]  Nikos Makris,et al.  Automatically parcellating the human cerebral cortex. , 2004, Cerebral cortex.

[5]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[6]  Thomas Baer,et al.  A model for the prediction of thresholds, loudness, and partial loudness , 1997 .

[7]  Gari D Clifford,et al.  Robust fundamental frequency estimation in sustained vowels: detailed algorithmic comparisons and information fusion with adaptive Kalman filtering. , 2014, The Journal of the Acoustical Society of America.

[8]  J. Simon,et al.  Cortical entrainment to continuous speech: functional roles and interpretations , 2014, Front. Hum. Neurosci..

[9]  Bruno L. Giordano,et al.  Abstract encoding of auditory objects in cortical activity patterns. , 2013, Cerebral cortex.

[10]  John J. Foxe,et al.  Resolving precise temporal processing properties of the auditory system using continuous stimuli. , 2009, Journal of neurophysiology.

[11]  J. Simon,et al.  Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. , 2012, Journal of neurophysiology.

[12]  B. Moore,et al.  A Model of Loudness Applicable to Time-Varying Sounds , 2002 .

[13]  Robert J. Zatorre,et al.  Spatial Localization after Excision of Human Auditory Cortex , 2001, The Journal of Neuroscience.

[14]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[15]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[16]  Robert J Zatorre,et al.  Neural specializations for speech and pitch: moving beyond the dichotomies , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[17]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[18]  Jonathan Z. Simon,et al.  Robust cortical entrainment to the speech envelope relies on the spectro-temporal fine structure , 2014, NeuroImage.

[19]  Ronald W. Schafer,et al.  Theory and Applications of Digital Speech Processing , 2010 .

[20]  John J. Foxe,et al.  At what time is the cocktail party? A late locus of selective attention to natural speech , 2012, The European journal of neuroscience.

[21]  Matthew H. Davis,et al.  Neural Oscillations Carry Speech Rhythm through to Comprehension , 2012, Front. Psychology.

[22]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[23]  Richard S. J. Frackowiak,et al.  Endogenous Cortical Rhythms Determine Cerebral Specialization for Speech Perception and Production , 2007, Neuron.

[24]  Christopher K. Kovach,et al.  Temporal Envelope of Time-Compressed Speech Represented in the Human Auditory Cortex , 2009, The Journal of Neuroscience.

[25]  Roy D. Patterson,et al.  A Dynamic Compressive Gammachirp Auditory Filterbank , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[26]  A. Boemio,et al.  Hierarchical and asymmetric temporal sensitivity in human auditory cortices , 2005, Nature Neuroscience.

[27]  A. Galaburda,et al.  Human Cerebral Cortex: Localization, Parcellation, and Morphometry with Magnetic Resonance Imaging , 1992, Journal of Cognitive Neuroscience.

[28]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[29]  J. Rauschecker,et al.  Phoneme and word recognition in the auditory ventral stream , 2012, Proceedings of the National Academy of Sciences.

[30]  Brian N. Pasley,et al.  Reconstructing Speech from Human Auditory Cortex , 2012, PLoS biology.

[31]  Roger B. H. Tootell,et al.  The advantage of combining MEG and EEG: Comparison to fMRI in focally stimulated visual cortex , 2007, NeuroImage.

[32]  T. Picton,et al.  Human Cortical Responses to the Speech Envelope , 2008, Ear and hearing.

[33]  Jorge E. Lopez de Cardenas,et al.  Collapse of a long cylindrical bubble , 1991 .

[34]  M. Corballis,et al.  Sound lateralization in subjects with callosotomy, callosal agenesis, or hemispherectomy. , 2005, Brain research. Cognitive brain research.

[35]  E. B. Newman,et al.  A Scale for the Measurement of the Psychological Magnitude Pitch , 1937 .

[36]  A. Engel,et al.  Fast propagating waves within the rodent auditory cortex. , 2011, Cerebral cortex.

[37]  R. Ilmoniemi,et al.  Interpreting magnetic fields of the brain: minimum norm estimates , 2006, Medical and Biological Engineering and Computing.

[38]  Nai Ding,et al.  Robust cortical encoding of slow temporal modulations of speech. , 2013, Advances in experimental medicine and biology.

[39]  Kuansan Wang,et al.  Auditory representations of acoustic signals , 1992, IEEE Trans. Inf. Theory.

[40]  Oded Ghitza,et al.  On the Role of Theta-Driven Syllabic Parsing in Decoding Speech: Intelligibility of Speech with a Manipulated Modulation Spectrum , 2012, Front. Psychology.

[41]  R. Patterson,et al.  The deterioration of hearing with age: frequency selectivity, the critical ratio, the audiogram, and speech threshold. , 1982, The Journal of the Acoustical Society of America.

[42]  David Poeppel,et al.  The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time' , 2003, Speech Commun..

[43]  David Poeppel,et al.  The Tracking of Speech Envelope in the Human Cortex , 2013, PloS one.

[44]  P. Morosan,et al.  Human Primary Auditory Cortex: Cytoarchitectonic Subdivisions and Mapping into a Spatial Reference System , 2001, NeuroImage.

[45]  Matthew H. Davis,et al.  Hierarchical Processing in Spoken Language Comprehension , 2003, The Journal of Neuroscience.

[46]  S. S. Stevens A scale for the measurement of a psychological magnitude: loudness. , 1936 .

[47]  Elaine J. Weyuker,et al.  Computability, complexity, and languages - fundamentals of theoretical computer science , 2014, Computer science and applied mathematics.

[48]  J. Obleser,et al.  Frequency modulation entrains slow neural oscillations and optimizes human listening behavior , 2012, Proceedings of the National Academy of Sciences.

[49]  Tatsuo K Sato,et al.  Traveling Waves in Visual Cortex , 2012, Neuron.

[50]  Mary F. Howard,et al.  Hemispheric asymmetry in mid and long latency neuromagnetic responses to single clicks , 2009, Hearing Research.

[51]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[52]  Roy D. Patterson,et al.  Direct Recordings of Pitch Responses from Human Auditory Cortex , 2010, Current Biology.

[53]  Robert A. Frazor,et al.  Standing Waves and Traveling Waves Distinguish Two Circuits in Visual Cortex , 2007, Neuron.

[54]  R. Patterson,et al.  Off-frequency listening and auditory-filter asymmetry. , 1980, The Journal of the Acoustical Society of America.

[55]  S. Taulu,et al.  Applications of the signal space separation method , 2005, IEEE Transactions on Signal Processing.

[56]  Alan C. Evans,et al.  MRI Atlas of the Human Cerebellum , 2000 .

[57]  D. Abrams,et al.  Right-Hemisphere Auditory Cortex Is Dominant for Coding Syllable Patterns in Speech , 2008, The Journal of Neuroscience.

[58]  Olaf Hauk,et al.  Comparison of noise-normalized minimum norm estimates for MEG analysis using multiple resolution metrics , 2011, NeuroImage.

[59]  E Ahissar,et al.  Speech comprehension is correlated with temporal response patterns recorded from auditory cortex , 2001, Proceedings of the National Academy of Sciences of the United States of America.