Neural Signatures of the Processing of Temporal Patterns in Sound

The ability to detect regularities in sound (i.e., recurring structure) is critical for effective perception, enabling, for example, change detection and prediction. Two seemingly unconnected lines of research concern the neural operations involved in processing regularities: one investigates how neural activity synchronizes with temporal regularities (e.g., frequency modulation; FM) in sounds, whereas the other focuses on increases in sustained activity during stimulation with repeating tone-frequency patterns. In three electroencephalography studies with male and female human participants, we investigated whether neural synchronization and sustained neural activity are dissociable, or whether they are functionally interdependent. Experiment I demonstrated that neural activity synchronizes with temporal regularity (FM) in sounds, and that sustained activity increases concomitantly. In Experiment II, phase coherence of FM in sounds was parametrically varied. Although neural synchronization was more sensitive to changes in FM coherence, such changes led to a systematic modulation of both neural synchronization and sustained activity, with magnitude increasing as coherence increased. In Experiment III, participants either performed a duration categorization task on the sounds, or a visual object tracking task to distract attention. Neural synchronization was observed regardless of task, whereas the sustained response was observed only when attention was on the auditory task, not under (visual) distraction. The results suggest that neural synchronization and sustained activity levels are functionally linked: both are sensitive to regularities in sounds. However, neural synchronization might reflect a more sensory-driven response to regularity, compared with sustained activity which may be influenced by attentional, contextual, or other experiential factors. SIGNIFICANCE STATEMENT Optimal perception requires that the auditory system detects regularities in sounds. Synchronized neural activity and increases in sustained neural activity both appear to index the detection of a regularity, but the functional interrelation of these two neural signatures is unknown. In three electroencephalography experiments, we measured both signatures concomitantly while listeners were presented with sounds containing frequency modulations that differed in their regularity. We observed that both neural signatures are sensitive to temporal regularity in sounds, although they functionally decouple when a listener is distracted by a demanding visual task. Our data suggest that neural synchronization reflects a more automatic response to regularity compared with sustained activity, which may be influenced by attentional, contextual, or other experiential factors.

[1]  T. Picton,et al.  Human auditory evoked potentials to frequency-modulated tones. , 1989, Ear and hearing.

[2]  M. Landy,et al.  The effect of viewpoint on perceived visual roughness. , 2007, Journal of vision.

[3]  T. Picton,et al.  The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. , 1987, Psychophysiology.

[4]  王亚周 Involvement of endoplasmic reticulum stress in the necroptosis ofmicroglia/macrophages after spinal cord injury. , 2015 .

[5]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[6]  Eero P. Simoncelli,et al.  Summary statistics in auditory perception , 2013, Nature Neuroscience.

[7]  S. Geisser,et al.  On methods in the analysis of profile data , 1959 .

[8]  David Poeppel,et al.  Low-Frequency Cortical Oscillations Entrain to Subthreshold Rhythmic Auditory Stimuli , 2017, The Journal of Neuroscience.

[9]  A. Oliva,et al.  Searching in dynamic displays: effects of configural predictability and spatiotemporal continuity. , 2007, Journal of vision.

[10]  I. Nelken,et al.  Modeling the auditory scene: predictive regularity representations and perceptual objects , 2009, Trends in Cognitive Sciences.

[11]  D. Rubin,et al.  r equivalent: A simple effect size indicator. , 2003, Psychological methods.

[12]  Anna C. Nobre,et al.  Anticipated moments: temporal structure in attention , 2017, Nature Reviews Neuroscience.

[13]  P. Cavanagh,et al.  Tracking multiple targets with multifocal attention , 2005, Trends in Cognitive Sciences.

[14]  Matthew H. Davis,et al.  Effortful Listening: The Processing of Degraded Speech Depends Critically on Attention , 2012, The Journal of Neuroscience.

[15]  Raymond J. Dolan,et al.  Outlier Responses Reflect Sensitivity to Statistical Structure in the Human Brain , 2013, PLoS Comput. Biol..

[16]  Adriane E. Seiffert,et al.  Attentional costs in multiple-object tracking , 2008, Cognition.

[17]  M. Chait,et al.  Neural Correlates of Auditory Figure-Ground Segregation Based on Temporal Coherence , 2016, Cerebral cortex.

[18]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[19]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[20]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[21]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[22]  Jonathan Z Simon,et al.  Cortical Representations of Speech in a Multitalker Auditory Scene , 2017, The Journal of Neuroscience.

[23]  Björn Herrmann,et al.  Attentional State Modulates the Effect of an Irrelevant Stimulus Dimension on Perception , 2018, Journal of experimental psychology. Human perception and performance.

[24]  M. Masson Using confidence intervals for graphically based data interpretation. , 2003, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[25]  E. Schröger Mismatch Negativity: A Microphone into Auditory Memory , 2007 .

[26]  C. Schroeder,et al.  The Spectrotemporal Filter Mechanism of Auditory Selective Attention , 2013, Neuron.

[27]  Jordi Costa-Faidella,et al.  Selective entrainment of brain oscillations drives auditory perceptual organization , 2017, NeuroImage.

[28]  Terence W Picton,et al.  Auditory Steady-State Responses to Exponential Modulation Envelopes , 2002, Ear and hearing.

[29]  Edward L. Bartlett,et al.  Ageing affects dual encoding of periodicity and envelope shape in rat inferior colliculus neurons , 2017, The European journal of neuroscience.

[30]  J. Simon,et al.  Evidence of degraded representation of speech in noise, in the aging midbrain and cortex. , 2016, Journal of neurophysiology.

[31]  Terence W. Picton,et al.  Multiple Auditory Steady-State Responses to AM and FM Stimuli , 2001, Audiology and Neurotology.

[32]  Matthew H. Davis,et al.  Dissociating speech perception and comprehension at reduced levels of awareness , 2007, Proceedings of the National Academy of Sciences.

[33]  Joachim Gross,et al.  Auditory cortical delta-entrainment interacts with oscillatory power in multiple fronto-parietal networks , 2017, NeuroImage.

[34]  P. Finkenzeller,et al.  Akustischen Reizen zugeordnete Gleichspannungsänderungen am intakten Schädel des Menschen , 2004, Pflügers Archiv.

[35]  J. Obleser,et al.  Frequency modulation entrains slow neural oscillations and optimizes human listening behavior , 2012, Proceedings of the National Academy of Sciences.

[36]  C. Schroeder,et al.  Low-frequency neuronal oscillations as instruments of sensory selection , 2009, Trends in Neurosciences.

[37]  Mounya Elhilali,et al.  Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration , 2010, The Journal of Neuroscience.

[38]  B. Hangya,et al.  Phase Entrainment of Human Delta Oscillations Can Mediate the Effects of Expectation on Reaction Speed , 2010, The Journal of Neuroscience.

[39]  Laura C. Dilley,et al.  Long-Term Temporal Tracking of Speech Rate Affects Spoken-Word Recognition , 2014, Psychological science.

[40]  M. Jones,et al.  Temporal Aspects of Stimulus-Driven Attending in Dynamic Arrays , 2002, Psychological science.

[41]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[42]  G. Karmos,et al.  Entrainment of Neuronal Oscillations as a Mechanism of Attentional Selection , 2008, Science.

[43]  Matthew H. Davis,et al.  Neural Oscillations Carry Speech Rhythm through to Comprehension , 2012, Front. Psychology.

[44]  W D Keidel,et al.  [Sound correlated d-c changes from the intact head of human subjects]. , 1969, Pflugers Archiv : European journal of physiology.

[45]  F. Varela,et al.  Measuring phase synchrony in brain signals , 1999, Human brain mapping.

[46]  Björn Herrmann,et al.  Low-Frequency Neural Oscillations Support Dynamic Attending in Temporal Context , 2014 .

[47]  B. Scholl What Have We Learned about Attention from Multiple-Object Tracking (and Vice Versa)? , 2009 .

[48]  E. Schröger,et al.  The detection of higher-order acoustic transitions is reflected in the N1 ERP. , 2018, Psychophysiology.

[49]  Josh H. McDermott,et al.  Sound Segregation via Embedded Repetition Is Robust to Inattention , 2015, Journal of experimental psychology. Human perception and performance.

[50]  I. Winkler,et al.  The role of attention in the formation of auditory streams , 2007, Perception & psychophysics.

[51]  J. Mills,et al.  The frequency-modulation following response in young and aged human subjects , 2002, Hearing Research.

[52]  Isabelle Peretz,et al.  Tagging the Neuronal Entrainment to Beat and Meter , 2011, The Journal of Neuroscience.

[53]  J. Simon,et al.  Emergence of neural encoding of auditory objects while listening to competing speakers , 2012, Proceedings of the National Academy of Sciences.

[54]  T. Baldeweg Repetition effects to sounds: evidence for predictive coding in the auditory system , 2006, Trends in Cognitive Sciences.

[55]  J. Obleser,et al.  Aging affects the balance of neural entrainment and top-down neural modulation in the listening brain , 2017, Nature Communications.

[56]  Karl J. Friston,et al.  Is predictability salient? A study of attentional capture by auditory patterns , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[57]  J. Bakdash,et al.  Repeated Measures Correlation , 2017, Front. Psychol..

[58]  Robin A A Ince,et al.  Irregular Speech Rate Dissociates Auditory Cortical Entrainment, Evoked Responses, and Frontal Alpha , 2015, The Journal of Neuroscience.

[59]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[60]  M. Chait,et al.  Great expectations: Is there evidence for predictive coding in auditory cortex? , 2017, Neuroscience.

[61]  Kaori Idemaru,et al.  Word recognition reflects dimension-based statistical learning. , 2011, Journal of experimental psychology. Human perception and performance.

[62]  Sven L Mattys,et al.  Magnified Neural Envelope Coding Predicts Deficits in Speech Perception in Noise , 2017, The Journal of Neuroscience.

[63]  I. Johnsrude,et al.  Attentional Modulation of Envelope-Following Responses at Lower (93–109 Hz) but Not Higher (217–233 Hz) Modulation Rates , 2017, Journal of the Association for Research in Otolaryngology.

[64]  Ralph Barnes,et al.  Expectancy, Attention, and Time , 2000, Cognitive Psychology.

[65]  T. Picton,et al.  Human auditory sustained potentials. I. The nature of the response. , 1978, Electroencephalography and clinical neurophysiology.

[66]  Alexandra Bendixen,et al.  Predictability effects in auditory scene analysis: a review , 2014, Front. Neurosci..

[67]  Z W Pylyshyn,et al.  Tracking multiple independent targets: evidence for a parallel tracking mechanism. , 1988, Spatial vision.

[68]  W. Kohler,et al.  Currents of the human auditory cortex. , 1955, Journal of cellular physiology. Supplement.

[69]  M. Chait,et al.  Detecting and representing predictable structure during auditory scene analysis , 2016, eLife.

[70]  C. Schroeder,et al.  Predictive Suppression of Cortical Excitability and Its Deficit in Schizophrenia , 2013, The Journal of Neuroscience.

[71]  Saskia Haegens,et al.  Temporal expectations and neural amplitude fluctuations in auditory cortex interactively influence perception , 2016, NeuroImage.

[72]  Björn Herrmann,et al.  Oscillatory Phase Dynamics in Neural Entrainment Underpin Illusory Percepts of Time , 2013, The Journal of Neuroscience.

[73]  G. Studebaker A "rationalized" arcsine transform. , 1985, Journal of speech and hearing research.

[74]  Karl J. Friston,et al.  Brain responses in humans reveal ideal observer-like sensitivity to complex acoustic patterns , 2016, Proceedings of the National Academy of Sciences.

[75]  D. Poeppel,et al.  Cortical entrainment to music and its modulation by expertise , 2015, Proceedings of the National Academy of Sciences.

[76]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[77]  S. Shamma,et al.  Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene , 2009, PLoS biology.

[78]  E. Sussman Auditory Scene Analysis: An Attention Perspective. , 2017, Journal of speech, language, and hearing research : JSLHR.

[79]  J. Obleser,et al.  Entrained neural oscillations in multiple frequency bands comodulate behavior , 2014, Proceedings of the National Academy of Sciences.

[80]  N. Logothetis,et al.  Frequency-Band Coupling in Surface EEG Reflects Spiking Activity in Monkey Visual Cortex , 2009, Neuron.

[81]  R. Hari,et al.  Effect of stimulus repetition on negative sustained potentials elicited by auditory and visual stimuli in the human EEG , 1978, Biological Psychology.