Tracking the signal, cracking the code: speech and speech comprehension in non-invasive human electrophysiology

ABSTRACT Magneto- and electroencephalographic (M/EEG) signals recorded from the human scalp have allowed for substantial advances for neural models of speech comprehension over the past decades. These methods are currently advancing rapidly and continue to offer unparalleled insight in the near-to-real-time neural dynamics of speech processing. We provide a historically informed overview over dependent measures in the time and frequency domain and highlight recent advances resulting from these measures. We discuss the notorious challenges (and solutions) speech and language researchers are faced with when studying auditory brain responses in M/EEG. We argue that a key to understanding the neural basis of speech comprehension will lie in studying interactions between the neural tracking of speech and the functional neural network dynamics. This article is intended for both, non-experts who want to learn how to use M/EEG to study speech comprehension and scholars aiming for an overview of state-of-the-art M/EEG analysis methods.

[1]  J. Obleser,et al.  Spatiotemporal dynamics of auditory attention synchronize with speech , 2016, Proceedings of the National Academy of Sciences.

[2]  Benedikt Zoefel,et al.  EEG oscillations entrain their phase to high-level features of speech sound , 2016, NeuroImage.

[3]  J. Obleser,et al.  Temporal expectations and neural amplitude fluctuations in auditory cortex interactively influence perception , 2016, NeuroImage.

[4]  Alexander T Sack,et al.  Oscillatory phase shapes syllable perception , 2015, Proceedings of the National Academy of Sciences.

[5]  D. Poeppel,et al.  Cortical Tracking of Hierarchical Linguistic Structures in Connected Speech , 2015, Nature Neuroscience.

[6]  S. Debener,et al.  Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear , 2015, Scientific Reports.

[7]  Robin A A Ince,et al.  Irregular Speech Rate Dissociates Auditory Cortical Entrainment, Evoked Responses, and Frontal Alpha , 2015, The Journal of Neuroscience.

[8]  Edmund C. Lalor,et al.  Low-Frequency Cortical Entrainment to Speech Reflects Phoneme-Level Processing , 2015, Current Biology.

[9]  Ying-Yee Kong,et al.  Effects of Spectral Degradation on Attentional Modulation of Cortical Auditory Responses to Continuous Speech , 2015, Journal of the Association for Research in Otolaryngology.

[10]  Toralf Neuling,et al.  Friends, not foes: Magnetoencephalography as a tool to uncover brain dynamics during transcranial alternating current stimulation , 2015, NeuroImage.

[11]  I. Winkler,et al.  Auditory perceptual objects as generative models: Setting the stage for communication by sound , 2015, Brain and Language.

[12]  John J. Foxe,et al.  Attentional Selection in a Cocktail Party Environment Can Be Decoded from Single-Trial EEG. , 2015, Cerebral cortex.

[13]  J. Obleser,et al.  Slow-delta phase concentration marks improved temporal expectations based on the passage of time. , 2015, Psychophysiology.

[14]  J. Obleser,et al.  Alpha Oscillatory Dynamics Index Temporal Expectation Benefits in Working Memory. , 2015, Cerebral cortex.

[15]  Robin A. A. Ince,et al.  Frontal Top-Down Signals Increase Coupling of Auditory Low-Frequency Oscillations to Continuous Speech in Human Listeners , 2015, Current Biology.

[16]  Maarten De Vos,et al.  Decoding the attended speech stream with multi-channel EEG: implications for online, daily-life applications , 2015, Journal of neural engineering.

[17]  Erich Schröger,et al.  Acoustic Detail Guides Attention Allocation in a Selective Listening Task , 2015, Journal of Cognitive Neuroscience.

[18]  J. Macke,et al.  Neural population coding: combining insights from microscopic and mass signals , 2015, Trends in Cognitive Sciences.

[19]  Jonas Obleser,et al.  Alpha Phase Determines Successful Lexical Decision in Noise , 2015, The Journal of Neuroscience.

[20]  Björn Herrmann,et al.  Neural Alpha Dynamics in Younger and Older Listeners Reflect Acoustic Challenges and Predictive Benefits , 2015, The Journal of Neuroscience.

[21]  Olivier Bertrand,et al.  Oscillatory alpha modulations in right auditory regions reflect the validity of acoustic cues in an auditory spatial attention task. , 2014, Cerebral cortex.

[22]  Ying-Yee Kong,et al.  Differential modulation of auditory responses to attended and unattended speech in different listening conditions , 2014, Hearing Research.

[23]  J. Obleser,et al.  Entrained neural oscillations in multiple frequency bands comodulate behavior , 2014, Proceedings of the National Academy of Sciences.

[24]  Christopher J. Plack,et al.  Perceptual Consequences of “Hidden” Hearing Loss , 2014, Trends in hearing.

[25]  Matthew S. Tata,et al.  Theta-band phase tracking in the two-talker problem , 2014, Brain and Language.

[26]  M. Wibral,et al.  Untangling cross-frequency coupling in neuroscience , 2014, Current Opinion in Neurobiology.

[27]  J. Simon,et al.  Cortical entrainment to continuous speech: functional roles and interpretations , 2014, Front. Hum. Neurosci..

[28]  Jonas Obleser,et al.  Cortical alpha oscillations as a tool for auditory selective inhibition , 2014, Front. Hum. Neurosci..

[29]  O. Sporns Contributions and challenges for network models in cognitive neuroscience , 2014, Nature Neuroscience.

[30]  P. Uhlhaas,et al.  Working memory and neural oscillations: alpha–gamma versus theta–gamma codes for distinct WM information? , 2014, Trends in Cognitive Sciences.

[31]  F. D. Silva,et al.  EEG and MEG: Relevance to Neuroscience , 2013, Neuron.

[32]  C. Herrmann,et al.  Transcranial alternating current stimulation: a review of the underlying mechanisms and modulation of cognitive processes , 2013, Front. Hum. Neurosci..

[33]  J. Obleser,et al.  Frequency-specific adaptation in human auditory cortex depends on the spectral variance in the acoustic stimulation. , 2013, Journal of neurophysiology.

[34]  Jonathan Z. Simon,et al.  Adaptive Temporal Encoding Leads to a Background-Insensitive Cortical Representation of Speech , 2013, The Journal of Neuroscience.

[35]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[36]  J. Obleser,et al.  Frequency modulation entrains slow neural oscillations and optimizes human listening behavior , 2012, Proceedings of the National Academy of Sciences.

[37]  Matthew H. Davis,et al.  Neural Oscillations Carry Speech Rhythm through to Comprehension , 2012, Front. Psychology.

[38]  Burkhard Maess,et al.  Adverse Listening Conditions and Memory Load Drive a Common Alpha Oscillatory Network , 2012, The Journal of Neuroscience.

[39]  Hari M. Bharadwaj,et al.  Why Middle-Aged Listeners Have Trouble Hearing in Everyday Settings , 2012, Current Biology.

[40]  Scott Makeig,et al.  Familiarity with Speech Affects Cortical Processing of Auditory Distance Cues and Increases Acuity , 2012, PloS one.

[41]  J. Simon,et al.  Emergence of neural encoding of auditory objects while listening to competing speakers , 2012, Proceedings of the National Academy of Sciences.

[42]  Nathan Weisz,et al.  Lateralized auditory cortical alpha band activity and interregional connectivity pattern reflect anticipation of target sounds. , 2012, Cerebral cortex.

[43]  C. Koch,et al.  The origin of extracellular fields and currents — EEG, ECoG, LFP and spikes , 2012, Nature Reviews Neuroscience.

[44]  Joachim Gross,et al.  Phase-Locked Responses to Speech in Human Auditory Cortex are Enhanced During Comprehension , 2012, Cerebral cortex.

[45]  N. Mesgarani,et al.  Selective cortical representation of attended speaker in multi-talker speech perception , 2012, Nature.

[46]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[47]  Hermann Ackermann,et al.  Magnetic brain activity phase-locked to the envelope, the syllable onsets, and the fundamental frequency of a perceived speech signal. , 2012, Psychophysiology.

[48]  David Poeppel,et al.  A mutual information analysis of neural coding of speech by low-frequency MEG phase information. , 2011, Journal of neurophysiology.

[49]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[50]  Jonas Obleser,et al.  Alpha Rhythms in Audition: Cognitive and Clinical Perspectives , 2011, Front. Psychology.

[51]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[52]  R. Knight,et al.  The functional role of cross-frequency coupling , 2010, Trends in Cognitive Sciences.

[53]  O. Jensen,et al.  Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by Inhibition , 2010, Front. Hum. Neurosci..

[54]  Seppo P. Ahlfors,et al.  Sensitivity of MEG and EEG to Source Orientation , 2010, Brain Topography.

[55]  Till R. Schneider,et al.  Using ICA for the Analysis of Multi-Channel EEG Data , 2010 .

[56]  John J. Foxe,et al.  Neural responses to uninterrupted natural speech can be extracted with precise temporal resolution , 2010, The European journal of neuroscience.

[57]  Christopher K. Kovach,et al.  Temporal Envelope of Time-Compressed Speech Represented in the Human Auditory Cortex , 2009, The Journal of Neuroscience.

[58]  I. Nelken,et al.  Modeling the auditory scene: predictive regularity representations and perceptual objects , 2009, Trends in Cognitive Sciences.

[59]  M. Liberman,et al.  Adding Insult to Injury: Cochlear Nerve Degeneration after “Temporary” Noise-Induced Hearing Loss , 2009, The Journal of Neuroscience.

[60]  Thomas Lunner,et al.  Cognition and hearing aids. , 2009, Scandinavian journal of psychology.

[61]  Antoine J. Shahin,et al.  Brain oscillations during semantic evaluation of speech , 2009, Brain and Cognition.

[62]  A. Friederici,et al.  Syntactic learning by mere exposure - An ERP study in adult learners , 2009, BMC Neuroscience.

[63]  John J. Foxe,et al.  Resolving precise temporal processing properties of the auditory system using continuous stimuli. , 2009, Journal of neurophysiology.

[64]  I. Winkler,et al.  I Heard That Coming: Event-Related Potential Evidence for Stimulus-Driven Prediction in the Auditory System , 2009, The Journal of Neuroscience.

[65]  R. VanRullen,et al.  The Phase of Ongoing EEG Oscillations Predicts Visual Perception , 2009, The Journal of Neuroscience.

[66]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[67]  L. Vaina,et al.  Mapping the signal‐to‐noise‐ratios of cortical sources in magnetoencephalography and electroencephalography , 2009, Human brain mapping.

[68]  W. Klimesch,et al.  What does phase information of oscillatory brain activity tell us about cognitive processes? , 2008, Neuroscience & Biobehavioral Reviews.

[69]  Jonathan Z. Simon,et al.  Denoising based on spatial filtering , 2008, Journal of Neuroscience Methods.

[70]  A. Puce,et al.  Neuronal oscillations and visual amplification of speech , 2008, Trends in Cognitive Sciences.

[71]  R. Näätänen,et al.  The mismatch negativity (MMN) in basic research of central auditory processing: A review , 2007, Clinical Neurophysiology.

[72]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[73]  Mante S. Nieuwland,et al.  Establishing reference in language comprehension: An electrophysiological perspective , 2007, Brain Research.

[74]  A. Nakamura,et al.  Localizing the distributed language network responsible for the N400 measured by MEG during auditory sentence processing , 2006, Brain Research.

[75]  P. Olejniczak,et al.  Neurophysiologic Basis of EEG , 2006, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[76]  Seppo P. Ahlfors,et al.  Assessing and improving the spatial accuracy in MEG source localization by depth-weighted minimum-norm estimates , 2006, NeuroImage.

[77]  Ankoor S. Shah,et al.  An oscillatory hierarchy controlling neuronal excitability and stimulus processing in the auditory cortex. , 2005, Journal of neurophysiology.

[78]  A. Bruns Fourier-, Hilbert- and wavelet-based signal analysis: are they really different approaches? , 2004, Journal of Neuroscience Methods.

[79]  G. Buzsáki,et al.  Neuronal Oscillations in Cortical Networks , 2004, Science.

[80]  S. Makeig,et al.  Mining event-related brain dynamics , 2004, Trends in Cognitive Sciences.

[81]  P. Hagoort,et al.  Integration of Word Meaning and World Knowledge in Language Comprehension , 2004, Science.

[82]  Matthew H. Davis,et al.  Hierarchical Processing in Spoken Language Comprehension , 2003, The Journal of Neuroscience.

[83]  Anders M. Dale,et al.  N400-like Magnetoencephalography Responses Modulated by Semantic Context, Word Frequency, and Lexical Class in Sentences , 2002, NeuroImage.

[84]  M Kathleen Pichora-Fuller,et al.  Listening in aging adults: from discourse comprehension to psychoacoustics. , 2002, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[85]  G. R. Barnes,et al.  A Quantitative Assessment of the Sensitivity of Whole-Head MEG to Activity in the Adult Human Cortex , 2002, NeuroImage.

[86]  P. Chauvel,et al.  Neuromagnetic source localization of auditory evoked fields and intracerebral evoked potentials: a comparison of data in the same patients , 2001, Clinical Neurophysiology.

[87]  John J. Foxe,et al.  Attention-dependent suppression of distracter visual input can be cross-modally cued as indexed by anticipatory parieto-occipital alpha-band oscillations. , 2001, Brain research. Cognitive brain research.

[88]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[89]  Colin M. Brown,et al.  ERP effects of listening to speech: semantic ERP effects , 2000, Neuropsychologia.

[90]  Colin M. Brown,et al.  ERP effects of listening to speech compared to reading: the P600/SPS to syntactic violations in spoken sentences and rapid serial visual presentation , 2000, Neuropsychologia.

[91]  M. Whittington,et al.  Gamma and beta frequency oscillations in response to novel auditory stimuli: A comparison of human electroencephalogram (EEG) data with in vitro models. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[92]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[93]  M. Daneman,et al.  Listening to discourse in distracting settings: the effects of aging. , 2000, Psychology and aging.

[94]  Karl J. Friston,et al.  Multisubject fMRI Studies and Conjunction Analyses , 1999, NeuroImage.

[95]  A Wingfield,et al.  One voice too many: adult age differences in language processing with different types of distracting sounds. , 1999, The journals of gerontology. Series B, Psychological sciences and social sciences.

[96]  O. Bertrand,et al.  Oscillatory gamma activity in humans and its role in object representation , 1999, Trends in Cognitive Sciences.

[97]  Angela D. Friederici,et al.  Brain potentials indicate immediate use of prosodic cues in natural speech processing , 1999, Nature Neuroscience.

[98]  V. Samar,et al.  Wavelet Analysis of Neuroelectric Waveforms: A Conceptual Tutorial , 1999, Brain and Language.

[99]  R. Ilmoniemi,et al.  Combined mapping of human auditory EEG and MEG responses. , 1998, Electroencephalography and clinical neurophysiology.

[100]  B C Moore,et al.  Development and evaluation of a procedure for fitting multi-channel compression hearing aids. , 1998, British journal of audiology.

[101]  M. E. Spencer,et al.  A Study of Dipole Localization Accuracy for MEG and EEG using a Human Skull Phantom , 1998, NeuroImage.

[102]  S Makeig,et al.  Blind separation of auditory event-related brain responses into independent components. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[103]  Lee Osterhout,et al.  Event-related brain potentials and human language , 1997, Trends in Cognitive Sciences.

[104]  Riitta Salmelin,et al.  Evidence for reactive magnetic 10-Hz rhythm in the human auditory cortex , 1997, Neuroscience Letters.

[105]  A. Friederici,et al.  Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. , 1993, Brain research. Cognitive brain research.

[106]  A. Dale,et al.  Improved Localizadon of Cortical Activity by Combining EEG and MEG with MRI Cortical Surface Reconstruction: A Linear Approach , 1993, Journal of Cognitive Neuroscience.

[107]  H. Neville,et al.  Natural speech processing: An analysis using event-related brain potentials , 1991, Psychobiology.

[108]  M. Scherg,et al.  A Source Analysis of the Late Human Auditory Evoked Potentials , 1989, Journal of Cognitive Neuroscience.

[109]  M. Posner,et al.  Positron emission tomographic studies of the cortical anatomy of single-word processing , 1988, Nature.

[110]  R. Näätänen,et al.  Attention effects on auditory EPs as a function of inter-stimulus interval , 1981, Biological Psychology.

[111]  S. M. Marcus Acoustic determinants of perceptual center (P-center) location , 1981, Perception & psychophysics.

[112]  M. Kutas,et al.  Reading between the lines: Event-related brain potentials during natural sentence processing , 1980, Brain and Language.

[113]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[114]  Robert M. Chapman,et al.  Brain responses related to semantic meaning , 1978, Brain and Language.

[115]  J. Morton,et al.  Perceptual centers (P-centers). , 1976 .

[116]  J. T. Marsh,et al.  Contextual meaning effects on speech-evoked potentials. , 1973, Behavioral biology.

[117]  C. C. Wood,et al.  Auditory Evoked Potentials during Speech Perception , 1971, Science.

[118]  H. Levitt Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[119]  W. Roth,et al.  The effect of attention on the average evoked response to speech sounds. , 1970, Electroencephalography and clinical neurophysiology.

[120]  R. Caton The Electric Currents of the Brain , 1970 .

[121]  D. Cohen Magnetoencephalography: Evidence of Magnetic Fields Produced by Alpha-Rhythm Currents , 1968, Science.

[122]  H. Davis,et al.  The slow response of the human cortex to auditory stimuli: recovery process. , 1966, Electroencephalography and clinical neurophysiology.

[123]  M. Diamond,et al.  Case of Intrauterine Death Due to α-Thalassaemia , 1965 .

[124]  P. Davis Effects of acoustic stimuli on the waking human brain , 1939 .

[125]  H. Berger Über das Elektrenkephalogramm des Menschen , 1938, Archiv für Psychiatrie und Nervenkrankheiten.

[126]  W. Ziegler The Oxford Handbook Of Event Related Potential Components , 2016 .

[127]  Preben Kidmose,et al.  In-Ear EEG From Viscoelastic Generic Earpieces: Robust and Unobtrusive 24/7 Monitoring , 2016, IEEE Sensors Journal.

[128]  J. Simon,et al.  Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. , 2012, Journal of neurophysiology.

[129]  E. C. Cmm,et al.  on the Recognition of Speech, with , 2008 .

[130]  P. Hagoort,et al.  Oscillatory neuronal dynamics during language comprehension. , 2006, Progress in brain research.

[131]  Matti S. Hämäläinen,et al.  Functional localization based on measurements with a whole-head magnetometer system , 2005, Brain Topography.

[132]  Jun Soo Kwon,et al.  Optimized Individual Mismatch Negativity Source Localization Using a Realistic Head Model and the Talairach Coordinate System , 2004, Brain Topography.

[133]  F. Varela,et al.  Measuring phase synchrony in brain signals , 1999, Human brain mapping.

[134]  George Zouridakis,et al.  Magnetoencephalographic mapping of the language-specific cortex , 1999 .

[135]  T. Elbert,et al.  Specific tonotopic organizations of different areas of the human auditory cortex revealed by simultaneous magnetic and electric recordings. , 1995, Electroencephalography and clinical neurophysiology.

[136]  W. Ritter,et al.  Cortical evoked potentials elicited by real speech words and human sounds. , 1975, Electroencephalography and clinical neurophysiology.

[137]  E. Adrian,et al.  Brain Rhythms , 1944, Nature.