Object-based attention in complex, naturalistic auditory streams

In vision, objects have been described as the ‘units’ on which non-spatial attention operates in many natural settings. Here, we test the idea of object-based attention in the auditory domain within ecologically valid auditory scenes, composed of two spatially and temporally overlapping sound streams (speech signal vs. environmental soundscapes in Experiment 1 and two speech signals in Experiment 2). Top-down attention was directed to one or the other auditory stream by a non-spatial cue. To test for high-level, object-based attention effects we introduce an auditory repetition detection task in which participants have to detect brief repetitions of auditory objects, ruling out any possible confounds with spatial or feature-based attention. The participants’ responses were significantly faster and more accurate in the valid cue condition compared to the invalid cue condition, indicating a robust cue-validity effect of high-level, object-based auditory attention.

[1]  Michael F. Bunting,et al.  The cocktail party phenomenon revisited: The importance of working memory capacity , 2001, Psychonomic bulletin & review.

[2]  Karsten Specht,et al.  Attention and cognitive control networks assessed in a dichotic listening fMRI study , 2011, Brain and Cognition.

[3]  Josh H. McDermott.  The cocktail party problem , 2009, Current Biology.

[4]  Aren Jansen,et al.  Audio Set: An ontology and human-labeled dataset for audio events , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  R. Rafal,et al.  Shifting visual attention between objects and locations: evidence from normal and parietal lesion subjects. , 1994, Journal of experimental psychology. General.

[6]  Katherine M. Armstrong,et al.  Selective gating of visual signals by microstimulation of frontal cortex , 2003, Nature.

[7]  J. Findlay,et al.  Sensitivity and criterion effects in the spatial cuing of visual attention , 1987, Perception & psychophysics.

[8]  John T. Serences,et al.  Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices , 2013, Nature Neuroscience.

[9]  Taosheng Liu,et al.  Neural representation of object-specific attentional priority , 2016, NeuroImage.

[10]  Riitta Hari,et al.  Left Superior Temporal Gyrus Is Coupled to Attended Speech in a Cocktail-Party Auditory Scene , 2016, The Journal of Neuroscience.

[11]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[12]  Barbara Shinn-Cunningham,et al.  Spatial release from energetic and informational masking in a selective speech identification task. , 2008, The Journal of the Acoustical Society of America.

[13]  A. Engel,et al.  Neuronal Synchronization along the Dorsal Visual Pathway Reflects the Focus of Spatial Attention , 2008, Neuron.

[14]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[15]  A. Treisman.  Contextual Cues in Selective Listening , 1960 .

[16]  Adrian K. C. Lee,et al.  Using neuroimaging to understand the cortical mechanisms of auditory selective attention , 2014, Hearing Research.

[17]  DeLiang Wang,et al.  Isolating the energetic component of speech-on-speech masking with ideal time-frequency segregation. , 2006, The Journal of the Acoustical Society of America.

[18]  J. Werker,et al.  Tuned to the signal: the privileged status of speech for young infants. , 2004, Developmental science.

[19]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[20]  E. C. Cmm,et al.  on the Recognition of Speech, with , 2008 .

[21]  Lee M. Miller,et al.  Auditory attentional control and selection during cocktail party listening. , 2010, Cerebral cortex.

[22]  I. Winkler,et al.  Human auditory cortex tracks task-irrelevant sound sources , 2003, Neuroreport.

[23]  Larry S. Davis,et al.  Pitch and timbre manipulations using cortical representation of sound , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[24]  D. Kimura.  Left-right Differences in the Perception of Melodies , 1964 .

[25]  Barbara G. Shinn-Cunningham,et al.  Auditory Selective Attention Reveals Preparatory Activity in Different Cortical Regions for Selection Based on Source Location and Source Pitch , 2012, Front. Neurosci..

[26]  Sean T. Stevens,et al.  Comparing the time course and efficacy of spatial and feature-based attention , 2007, Vision Research.

[27]  Mounya Elhilali,et al.  A cocktail party with a cortical twist: how cortical mechanisms contribute to sound segregation. , 2008, The Journal of the Acoustical Society of America.

[28]  John H Reynolds,et al.  Object-based attention to one of two superimposed surfaces alters responses in human early visual cortex. , 2011, Journal of neurophysiology.

[29]  R. Desimone,et al.  High-Frequency, Long-Range Coupling Between Prefrontal and Visual Cortex During Attention , 2009, Science.

[30]  Daniel Baldauf,et al.  Top-down biasing signals of non-spatial, object-based attention. , 2015, Journal of vision.

[31]  J. Duncan.  Selective attention and the organization of visual information. , 1984, Journal of experimental psychology. General.

[32]  B. Shinn-Cunningham.  Object-based auditory and visual attention , 2008, Trends in Cognitive Sciences.

[33]  M. Paradiso,et al.  Feature-specific effects of selective visual attention , 1995, Vision Research.

[34]  G. Boynton,et al.  Global feature-based attention for motion and color , 2003, Vision Research.

[35]  S. Shamma,et al.  Temporal coherence and attention in auditory scene analysis , 2011, Trends in Neurosciences.

[36]  P. Verghese,et al.  Attention to Multiple Objects Facilitates Their Integration in Prefrontal and Parietal Cortex , 2017, The Journal of Neuroscience.

[37]  E. Yund,et al.  Attentional modulation of human auditory cortex , 2004, Nature Neuroscience.

[38]  Stefan Treue,et al.  Feature-based attention influences motion processing gain in macaque visual cortex , 1999, Nature.

[39]  Heiner Deubel,et al.  Attentional Selection of Multiple Goal Positions Before Rapid Hand Movement Sequences: An Event-related Potential Study , 2009, Journal of Cognitive Neuroscience.

[40]  Leslie G. Ungerleider,et al.  Attentional selection of multiple objects in the human visual system , 2017, NeuroImage.

[41]  Jonathan Z. Simon,et al.  Adaptive Temporal Encoding Leads to a Background-Insensitive Cortical Representation of Speech , 2013, The Journal of Neuroscience.

[42]  Teemu Rinne,et al.  Stimulus-dependent activations and attention-related modulations in the auditory cortex: A meta-analysis of fMRI studies , 2014, Hearing Research.

[43]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[44]  S A Hillyard,et al.  Feature-selective attention enhances color signals in early visual areas of the human brain , 2006, Proceedings of the National Academy of Sciences.

[45]  I. Winkler,et al.  The role of attention in the formation of auditory streams , 2007, Perception & psychophysics.

[46]  Barbara G. Shinn-Cunningham,et al.  Influence of Task-Relevant and Task-Irrelevant Feature Continuity on Selective Auditory Attention , 2012, Journal of the Association for Research in Otolaryngology.

[47]  F. Tong,et al.  Neural mechanisms of object-based attention. , 2015, Cerebral cortex.

[48]  Mounya Elhilali,et al.  Competing Streams at the Cocktail Party: Exploring the Mechanisms of Attention and Temporal Integration , 2010, The Journal of Neuroscience.

[49]  R. Zatorre,et al.  Structure and function of auditory cortex: music and speech , 2002, Trends in Cognitive Sciences.

[50]  S. Luck,et al.  Attention to Features Precedes Attention to Locations in Visual Search: Evidence from Electromagnetic Brain Responses in Humans , 2004, The Journal of Neuroscience.

[51]  John H. R. Maunsell,et al.  Feature-based attention in visual cortex , 2006, Trends in Neurosciences.

[52]  K. Hugdahl,et al.  The “Forced-Attention Paradigm” in Dichotic Listening to CV-Syllables: A Comparison Between Adults and Children , 1986, Cortex.

[53]  Josh H. McDermott,et al.  Attentive Tracking of Sound Sources , 2015, Current Biology.

[54]  Katrin Krumbholz,et al.  Feature- and Object-based Attentional Modulation in the Human Auditory Where Pathway , 2007, Journal of Cognitive Neuroscience.

[55]  Andrew R. A. Conway,et al.  Individual differences in working memory capacity and divided attention in dichotic listening , 2007, Psychonomic bulletin & review.

[56]  I. Nelken,et al.  Modeling the auditory scene: predictive regularity representations and perceptual objects , 2009, Trends in Cognitive Sciences.

[57]  Charles E. Schroeder,et al.  Motor contributions to the temporal precision of auditory attention , 2014, Nature Communications.

[58]  M. Corbetta,et al.  Neural basis and recovery of spatial attention deficits in spatial neglect , 2005, Nature Neuroscience.

[59]  Hari M. Bharadwaj,et al.  Bottom-up influences of voice continuity in focusing selective auditory attention , 2014, Psychological research.

[60]  I. Nelken,et al.  Processing of complex stimuli and natural scenes in the auditory cortex , 2004, Current Opinion in Neurobiology.

[61]  Nancy Kanwisher,et al.  fMRI evidence for objects as the units of attentional selection , 1999, Nature.

[62]  A. Kreiter,et al.  Feature-based attention and the suppression of non-relevant object features , 2008, Vision Research.

[63]  Virginia Best,et al.  Binaural interference and auditory grouping. , 2007, The Journal of the Acoustical Society of America.

[64]  Lee M. Miller,et al.  Tuning In to Sound: Frequency-Selective Attentional Filter in Human Primary Auditory Cortex , 2013, The Journal of Neuroscience.

[65]  H. Deubel,et al.  Attentional landscapes in reaching and grasping , 2010, Vision Research.

[66]  Adam Gazzaley,et al.  Preparatory encoding of the fine scale of human spatial attention , 2016, bioRxiv.

[67]  J. Simon,et al.  Emergence of neural encoding of auditory objects while listening to competing speakers , 2012, Proceedings of the National Academy of Sciences.

[68]  Sylvain Baillet,et al.  Motor origin of temporal predictions in auditory attention , 2017, Proceedings of the National Academy of Sciences.

[69]  Søren K. Andersen,et al.  Effects of Feature-selective and Spatial Attention at Different Stages of Visual Processing , 2011, Journal of Cognitive Neuroscience.

[70]  S. Shamma,et al.  Interaction between Attention and Bottom-Up Saliency Mediates the Representation of Foreground and Background in an Auditory Scene , 2009, PLoS biology.

[71]  Kenneth Hugdahl,et al.  Attention-related modulation of auditory-cortex responses to speech sounds during dichotic listening , 2012, Brain Research.

[72]  K. Hugdahl,et al.  Attention and cognitive control: unfolding the dichotic listening story. , 2009, Scandinavian journal of psychology.

[73]  Albert S. Bregman,et al.  The Auditory Scene. (Book Reviews: Auditory Scene Analysis. The Perceptual Organization of Sound.) , 1990 .

[74]  I. Winkler,et al.  Recording Event-Related Brain Potentials: Application to Study Auditory Perception , 2012 .

[75]  D. Poeppel,et al.  Speech perception at the interface of neurobiology and linguistics , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[76]  Virginia Best,et al.  Object continuity enhances selective auditory attention , 2008, Proceedings of the National Academy of Sciences.

[77]  R. Desimone,et al.  Transcranial alternating current stimulation (tACS) reveals causal role of brain oscillations in visual attention , 2016 .

[78]  T. Rinne,et al.  Selective attention to sound location or pitch studied with event‐related brain potentials and magnetic fields , 2008, The European journal of neuroscience.

[79]  Y. Cohen,et al.  The what, where and how of auditory-object perception , 2013, Nature Reviews Neuroscience.

[80]  R. Desimone,et al.  Alpha and gamma neurofeedback reinforce control of spatial attention , 2017 .

[81]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[82]  Hans-Jochen Heinze,et al.  Object-based attention involves the sequential activation of feature-specific cortical modules , 2014, Nature Neuroscience.

[83]  S. Hillyard,et al.  Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. , 1991, Journal of experimental psychology. Human perception and performance.

[84]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[85]  D. Gitelman,et al.  Covert Visual Spatial Orienting and Saccades: Overlapping Neural Systems , 2000, NeuroImage.

[86]  S. Luck,et al.  Feature-based attention modulates feedforward visual processing , 2009, Nature Neuroscience.

[87]  R. Zatorre,et al.  Voice-selective areas in human auditory cortex , 2000, Nature.

[88]  Justin Salamon,et al.  A Dataset and Taxonomy for Urban Sound Research , 2014, ACM Multimedia.

[89]  Luc H. Arnal,et al.  Asymmetric Function of Theta and Gamma Activity in Syllable Processing: An Intra-Cortical Study , 2012, Front. Psychology.

[90]  D. Kimura.  Functional Asymmetry of the Brain in Dichotic Listening , 1967 .

[91]  C Alain,et al.  Selectively attending to auditory objects. , 2000, Frontiers in bioscience : a journal and virtual library.

[92]  J. Simon,et al.  Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. , 2012, Journal of neurophysiology.

[93]  R. Carlyon.  How the brain separates sounds , 2004, Trends in Cognitive Sciences.

[94]  Jonathan Z Simon,et al.  The encoding of auditory objects in auditory cortex: insights from magnetoencephalography. , 2015, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[95]  H Stanislaw,et al.  Calculation of signal detection theory measures , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[96]  B. Scholl.  Objects and attention: the state of the art , 2001, Cognition.

[97]  D. Woods,et al.  Feature processing during high-rate auditory selective attention , 1993, Perception & psychophysics.

[98]  S. Jaeggi,et al.  The concurrent validity of the N-back task as a working memory measure , 2010, Memory.

[99]  M. Corbetta,et al.  A Common Network of Functional Areas for Attention and Eye Movements , 1998, Neuron.

[100]  Mikko Sams,et al.  Attention-driven auditory cortex short-term plasticity helps segregate relevant sounds from noise , 2011, Proceedings of the National Academy of Sciences.

[101]  Adrian K. C. Lee,et al.  Auditory selective attention is enhanced by a task-irrelevant temporally coherent visual stimulus in human listeners , 2015, eLife.

[102]  C. Darwin.  Auditory grouping , 1997, Trends in Cognitive Sciences.

[103]  Jonathan Z. Simon,et al.  Robust cortical entrainment to the speech envelope relies on the spectro-temporal fine structure , 2014, NeuroImage.

[104]  Viola S. Störmer,et al.  Feature-Based Attention Elicits Surround Suppression in Feature Space , 2014, Current Biology.

[105]  T. Griffiths,et al.  What is an auditory object? , 2004, Nature Reviews Neuroscience.