Representation of Auditory Motion Directions and Sound Source Locations in the Human Planum Temporale

The ability to precisely compute the location and direction of sounds in external space is a crucial perceptual process to efficiently interact with dynamic environments. Little is known, however, about how the human brain implements spatial hearing. In our study, we used fMRI to characterize the brain activity of humans listening to left, right, up and down moving as well as static sounds. Whole brain univariate results contrasting moving and static sounds varying in their location revealed a robust functional preference for auditory motion in bilateral human Planum Temporale (hPT). Importantly, multivariate pattern classification analysis showed that hPT contains information about both auditory motion directions and, to a lesser extent, sound source locations. More precisely, we observed that our classifier successfully decoded opposite axes of motion (vertical versus horizontal) but was less able to classify opposite within-axis direction (left versus right or up versus down); reminiscent of the axis of motion organization observed in the middle-temporal cortex for vision. Further multivariate analyses demonstrated that even if motion direction and location rely on partially shared pattern geometries in PT, the responses elicited by static and moving sounds were however highly distinct. Altogether our results demonstrate that human PT codes for auditory motion and location but that the underlying neural computation linked to motion processing is more reliable and partially distinct from the one supporting sound source location.

[1]  Yasuhito Sawahata,et al.  Spatial smoothing hurts localization but not information: Pitfalls for brain mappers , 2010, NeuroImage.

[2]  Jeremy Freeman,et al.  Orientation Decoding Depends on Maps, Not Columns , 2011, The Journal of Neuroscience.

[3]  Michael F. Neelon,et al.  The effect of trajectory on the auditory motion aftereffect , 2003, Hearing Research.

[4]  W. Singer,et al.  Auditory motion direction encoding in auditory cortex and high‐level visual cortex , 2012, Human brain mapping.

[5]  J. Lewald,et al.  The effect of spatial adaptation on auditory motion processing , 2011, Hearing Research.

[6]  M. Hoffmann,et al.  Direction‐specific adaptation of motion‐onset auditory evoked potentials , 2013, The European journal of neuroscience.

[7]  D. Moore,et al.  Auditory Neuroscience: The Salience of Looming Sounds , 2003, Current Biology.

[8]  J. Rauschecker,et al.  Mechanisms and streams for processing of "what" and "where" in auditory cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Giancarlo Valente,et al.  Active Sound Localization Sharpens Spatial Tuning in Human Primary Auditory Cortex , 2018, The Journal of Neuroscience.

[10]  Katrin Krumbholz,et al.  Hierarchical processing of sound location and motion in the human brainstem and planum temporale , 2005, The European journal of neuroscience.

[11]  M. W. Spitzer,et al.  Responses of inferior colliculus neurons to time-varying interaural phase disparity: effects of shifting the locus of virtual motion. , 1993, Journal of neurophysiology.

[12]  D. A. Benson,et al.  Single-unit activity in the auditory cortex of monkeys actively localizing sound sources: Spatial tuning and behavioral dependency , 1981, Brain Research.

[13]  Andreas Bartels,et al.  fMRI and its interpretations: an illustration on directional selectivity in area V5/MT , 2008, Trends in Neurosciences.

[14]  R Todd Constable,et al.  Image distortion correction in EPI: Comparison of field mapping with point spread function mapping , 2002, Magnetic resonance in medicine.

[15]  Gregory Hickok,et al.  An event-related fMRI study of auditory motion perception: No evidence for a specialized cortical system , 2007, Brain Research.

[16]  J. C. Middlebrooks,et al.  Location Coding by Opponent Neural Populations in the Auditory Cortex , 2005, PLoS biology.

[17]  J. Haynes Brain Reading: Decoding Mental States From Brain Activity In Humans , 2011 .

[18]  M. Abeles,et al.  Functional architecture in cat primary auditory cortex: columnar organization and organization according to depth. , 1970, Journal of neurophysiology.

[19]  J Blauert,et al.  Binaural localization. , 1982, Scandinavian audiology. Supplementum.

[20]  A. Caramazza,et al.  Brain Regions That Represent Amodal Conceptual Knowledge , 2013, The Journal of Neuroscience.

[21]  Eli Brenner,et al.  Time course of the effect of the Muller-Lyer illusion on saccades and perceptual judgments. , 2014, Journal of vision.

[22]  Simon B. Eickhoff,et al.  Assignment of functional activations to probabilistic cytoarchitectonic areas revisited , 2007, NeuroImage.

[23]  B. Shinn-Cunningham,et al.  Task-modulated “what” and “where” pathways in human auditory cortex , 2006, Proceedings of the National Academy of Sciences.

[24]  M. Sutter Spectral processing in the auditory cortex. , 2005, International review of neurobiology.

[25]  E. R. Cohen,et al.  Close correlation between activity in brain area MT/V5 and the perception of a visual motion aftereffect , 1998, Current Biology.

[26]  T. Griffiths,et al.  Distinct Mechanisms for Processing Spatial Sequences and Pitch Sequences in the Human Auditory Brain , 2003, The Journal of Neuroscience.

[27]  H. Barlow,et al.  Evidence for a Physiological Explanation of the Waterfall Phenomenon and Figural After-effects , 1963, Nature.

[28]  Ricardo Gattass,et al.  Electrophysiological Imaging of Functional Architecture in the Cortical Middle Temporal Visual Area of Cebus apella Monkey , 2003, The Journal of Neuroscience.

[29]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[30]  G L Romani,et al.  Human brain activation during passive listening to sounds from different locations: An fMRI and MEG study , 2005, Human brain mapping.

[31]  K. Jambrosic,et al.  Sound localization , 2004, Proceedings. Elmar-2004. 46th International Symposium on Electronics in Marine.

[32]  J. Rauschecker,et al.  Evidence for distinct human auditory cortex regions for sound location versus identity processing , 2013, Nature Communications.

[33]  Xiaoqin Wang,et al.  Level dependence of spatial processing in the primate auditory cortex. , 2012, Journal of neurophysiology.

[34]  Nikos K. Logothetis,et al.  Widespread and Opponent fMRI Signals Represent Sound Location in Macaque Auditory Cortex , 2017, Neuron.

[35]  N. Kriegeskorte,et al.  Author ' s personal copy Representational geometry : integrating cognition , computation , and the brain , 2013 .

[36]  E. Formisano,et al.  Opponent Coding of Sound Location (Azimuth) in Planum Temporale is Robust to Sound-Level Variations , 2015, Cerebral cortex.

[37]  B. Grothe,et al.  Mechanisms of sound localization in mammals. , 2010, Physiological reviews.

[38]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[39]  M. Cynader,et al.  The auditory motionaftereffect: Its tuning and specificity in the spatial and frequency domains , 2000 .

[40]  L. Aitkin,et al.  Azimuthal sensitivity of neurons in primary auditory cortex of cats. II. Organization along frequency-band strips. , 1990, Journal of neurophysiology.

[41]  O. Braddick,et al.  Brain Areas Sensitive to Coherent Visual Motion , 2001, Perception.

[42]  Jeremy Freeman,et al.  Coarse-Scale Biases for Spirals and Orientation in Human Visual Cortex , 2013, The Journal of Neuroscience.

[43]  G H Recanzone,et al.  Spatial processing in the auditory cortex of the macaque monkey. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[44]  D W Grantham,et al.  Detection and discrimination of simulated motion of auditory targets in the horizontal plane. , 1986, The Journal of the Acoustical Society of America.

[45]  Hans-Jochen Heinze,et al.  A movement-sensitive area in auditory cortex , 1999, Nature.

[46]  J. Saunders,et al.  Sensitivity to simulated directional sound motion in the rat primary auditory cortex. , 1999, Journal of neurophysiology.

[47]  J. C. Middlebrooks,et al.  Functional classes of neurons in primary auditory cortex of the cat distinguished by sensitivity to sound location , 1981, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[48]  E. Macaluso,et al.  A Common Cortical Substrate Activated by Horizontal and Vertical Sound Movement in the Human Brain , 2002, Current Biology.

[49]  Richard S. J. Frackowiak,et al.  Analysis of temporal structure in sound by the human brain , 1998, Nature Neuroscience.

[50]  Maarten Kamermans,et al.  A novel mechanism of cone photoreceptor adaptation , 2017, PLoS biology.

[51]  A. Dale,et al.  Visual motion aftereffect in human cortical area MT revealed by functional magnetic resonance imaging , 1995, Nature.

[52]  Hans P. Op de Beeck,et al.  Against hyperacuity in brain reading: Spatial smoothing does not hurt multivariate fMRI analyses? , 2010, NeuroImage.

[53]  Ranjan Batra,et al.  Topography of Interaural Temporal Disparity Coding in Projections of Medial Superior Olive to Inferior Colliculus , 2003, The Journal of Neuroscience.

[54]  D. Grantham,et al.  Auditory motion aftereffects , 1979, Perception & psychophysics.

[55]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[56]  R. Goebel,et al.  Mapping the Organization of Axis of Motion Selective Features in Human Area MT Using High-Field fMRI , 2011, PloS one.

[57]  Christoph M. Michel,et al.  Segregated Processing of Auditory Motion and Auditory Location: An ERP Mapping Study , 2002, NeuroImage.

[58]  Stefan Pollmann,et al.  The effect of acquisition resolution on orientation decoding from V1 BOLD fMRI at 7T , 2016, NeuroImage.

[59]  Ione Fine,et al.  Early Blindness Results in Developmental Plasticity for Auditory Motion Processing within Auditory and Occipital Cortex , 2016, Front. Hum. Neurosci..

[60]  T. Imig,et al.  Directionality derived from pinna-cue spectral notches in cat dorsal cochlear nucleus. , 2000, Journal of neurophysiology.

[61]  Sebastián M. Real,et al.  E2F1 Regulates Cellular Growth by mTORC1 Signaling , 2011, PloS one.

[62]  J. Kaas,et al.  Auditory processing in primate cerebral cortex , 1999, Current Opinion in Neurobiology.

[63]  Denis Schluppeck,et al.  Contribution of large scale biases in decoding of direction-of-motion from high-resolution fMRI data in human early visual cortex , 2012, NeuroImage.

[64]  R. Bowtell,et al.  “sparse” temporal sampling in auditory fMRI , 1999, Human brain mapping.

[65]  F. Tong,et al.  Decoding Seen and Attended Motion Directions from Activity in the Human Visual Cortex , 2006, Current Biology.

[66]  M. Ahissar,et al.  Encoding of sound-source location and movement: activity of single neurons and interactions between adjacent neurons in the monkey auditory cortex. , 1992, Journal of neurophysiology.

[67]  S. Edelman,et al.  Toward direct visualization of the internal shape representation space by fMRI , 1998, Psychobiology.

[68]  Wayne L. Neale,et al.  The effect of burst duration, interstimulus onset interval, and loudspeaker arrangement on auditory apparent motion in the free field. , 1994, The Journal of the Acoustical Society of America.

[69]  Yi Chen,et al.  Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): Random permutations and cluster size control , 2011, NeuroImage.

[70]  Hinze Hogendoorn,et al.  Decoding the motion aftereffect in human visual cortex , 2011 .

[71]  R. Zatorre,et al.  Spectral and temporal processing in human auditory cortex. , 2001, Cerebral cortex.

[72]  S. Anstis,et al.  Auditory adaptation to gradual rise or fall in intensity of a tone , 1982, Perception & psychophysics.

[73]  K. H. Britten,et al.  A relationship between behavioral choice and the visual responses of neurons in macaque MT , 1996, Visual Neuroscience.

[74]  E D Young,et al.  Neural organization and responses to complex stimuli in the dorsal cochlear nucleus. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[75]  James V. Haxby,et al.  CoSMoMVPA: Multi-Modal Multivariate Pattern Analysis of Neuroimaging Data in Matlab/GNU Octave , 2016, bioRxiv.

[76]  A. Rees,et al.  Auditory motion-specific mechanisms in the primate brain , 2017, PLoS biology.

[77]  D. M. Green,et al.  Sound localization by human listeners. , 1991, Annual review of psychology.

[78]  Doug J. K. Barrett,et al.  Response preferences for “what” and “where” in human non-primary auditory cortex , 2006, NeuroImage.

[79]  Franco Lepore,et al.  Positional, directional and speed selectivities in the primary auditory cortex of the cat , 1997, Hearing Research.

[80]  M. Cynader,et al.  Neurons in cat primary auditory cortex sensitive to correlates of auditory motion in three-dimensional space , 2005, Experimental Brain Research.

[81]  Patricia S. Goldman-Rakic,et al.  Reply to '‘What’, ‘where’ and ‘how’ in auditory cortex' , 2000, Nature Neuroscience.

[82]  T. Imig,et al.  Single-unit selectivity to azimuthal direction and sound pressure level of noise bursts in cat high-frequency primary auditory cortex. , 1990, Journal of neurophysiology.

[83]  J. C. Middlebrooks,et al.  Spatial Stream Segregation by Auditory Cortical Neurons , 2013, The Journal of Neuroscience.

[84]  Russell A. Poldrack,et al.  Large-scale automated synthesis of human functional neuroimaging data , 2011, Nature Methods.

[85]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[86]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.

[87]  Adam Bednar,et al.  Different spatio‐temporal electroencephalography features drive the successful decoding of binaural and monaural cues for sound localization , 2017, The European journal of neuroscience.

[88]  D. Heeger,et al.  Motion Opponency in Visual Cortex , 1999, The Journal of Neuroscience.

[89]  T. Imig,et al.  Functional organization of sound direction and sound pressure level in primary auditory cortex of the cat. , 1994, Journal of neurophysiology.

[90]  Adriane E. Seiffert,et al.  Direction-selective patterns of activity in human visual cortex suggest common neural substrates for different types of motion , 2012, Neuropsychologia.

[91]  J. Rauschecker,et al.  Functional Specialization in Rhesus Monkey Auditory Cortex , 2001, Science.

[92]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[93]  L P SANCHEZ-LONGO,et al.  Clinical Significance of Impairment of Sound Localization , 1958, Neurology.

[94]  R. Zatorre,et al.  Where is 'where' in the human auditory cortex? , 2002, Nature Neuroscience.

[95]  Frank Tong,et al.  Radial bias is not necessary for orientation decoding , 2016, NeuroImage.

[96]  Lee M. Miller,et al.  Populations of auditory cortical neurons can accurately encode acoustic space across stimulus intensity , 2009, Proceedings of the National Academy of Sciences.

[97]  B J Geesaman,et al.  Maps of complex motion selectivity in the superior temporal cortex of the alert macaque monkey: a double-label 2-deoxyglucose study. , 1997, Cerebral cortex.

[98]  Giancarlo Valente,et al.  The effect of spatial resolution on decoding accuracy in fMRI multivariate pattern analysis , 2016, NeuroImage.

[99]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[100]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[101]  J. C. Middlebrooks Sound localization. , 2015, Handbook of clinical neurology.

[102]  Karin A. Buetler,et al.  Electrical neuroimaging during auditory motion aftereffects reveals that auditory motion processing is motion sensitive but not direction selective. , 2013, Journal of neurophysiology.

[103]  Philipp Sterzer,et al.  Decoding pattern motion information in V1 , 2013, Cortex.

[104]  John C. Middlebrooks,et al.  Auditory space processing: here, there or everywhere? , 2002, Nature Neuroscience.

[105]  Ione Fine,et al.  Auditory motion processing after early blindness. , 2014, Journal of vision.

[106]  R. Desimone,et al.  Columnar organization of directionally selective cells in visual area MT of the macaque. , 1984, Journal of neurophysiology.

[107]  H S Colburn,et al.  Model for auditory localization. , 1976, The Journal of the Acoustical Society of America.

[108]  K. H. Britten,et al.  Motion adaptation in area MT. , 2002, Journal of neurophysiology.

[109]  D. Heeger,et al.  Neuronal Basis of the Motion Aftereffect Reconsidered , 2001, Neuron.

[110]  M. Mishkin,et al.  Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex , 1999, Nature Neuroscience.

[111]  Norimichi Kitagawa,et al.  Hearing visual motion in depth , 2002, Nature.

[112]  D. McAlpine,et al.  A neural code for low-frequency sound localization in mammals , 2001, Nature Neuroscience.

[113]  J. Rauschecker,et al.  Perception of Sound-Source Motion by the Human Brain , 2002, Neuron.

[114]  A. Vanlierde,et al.  Specific activation of the V5 brain area by auditory motion processing: an fMRI study. , 2005, Brain research. Cognitive brain research.

[115]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[116]  Franco Lepore,et al.  Auditory motion in the sighted and blind: Early visual deprivation triggers a large-scale imbalance between auditory and “visual” brain regions , 2016, NeuroImage.

[117]  Bertrand Delgutte,et al.  Decoding Sound Source Location and Separation Using Neural Population Activity Patterns , 2013, The Journal of Neuroscience.

[118]  Gregory Hickok,et al.  Human cortical auditory motion areas are not motion selective , 2004, Neuroreport.

[119]  Rudolf Stark,et al.  Imagined and Executed Actions in the Human Motor System: Testing Neural Similarity Between Execution and Imagery of Actions with a Multivariate Approach , 2016, Cerebral cortex.

[120]  M. Cynader,et al.  The auditory motion aftereffect: its tuning and specificity in the spatial and frequency domains. , 2000, Perception & psychophysics.

[121]  Gregory Hickok,et al.  Auditory Spatial and Object Processing in the Human Planum Temporale: No Evidence for Selectivity , 2010, Journal of Cognitive Neuroscience.

[122]  J. Boudreau,et al.  Binaural interaction in the cat superior olive S segment. , 1967, Journal of neurophysiology.

[123]  C. Grady,et al.  “What” and “where” in the human auditory system , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[124]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[125]  Jörg Lewald,et al.  Processing of sound location in human cortex , 2008, The European journal of neuroscience.

[126]  J. Thiran,et al.  Distinct Pathways Involved in Sound Recognition and Localization: A Human fMRI Study , 2000, NeuroImage.

[127]  Anthony J. Movshon,et al.  Visual Response Properties of Striate Cortical Neurons Projecting to Area MT in Macaque Monkeys , 1996, The Journal of Neuroscience.

[128]  Simon Carlile,et al.  The Perception of Auditory Motion , 2016, Trends in hearing.

[129]  Jörg Lewald,et al.  Cortical processing of change in sound location: Smooth motion versus discontinuous displacement , 2012, Brain Research.

[130]  M. Cynader,et al.  Auditory cortex neurons sensitive to correlates of auditory motion: underlying mechanisms , 2005, Experimental Brain Research.

[131]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[132]  D McAlpine,et al.  Spatial receptive fields of inferior colliculus neurons to auditory apparent motion in free field. , 2001, Journal of neurophysiology.

[133]  Neil W. Roach,et al.  Distortions of perceived auditory and visual space following adaptation to motion , 2008, Experimental Brain Research.

[134]  S. Clarke,et al.  Auditory spatial deficits following hemispheric lesions: Dissociation of explicit and implicit processing , 2012, Neuropsychological rehabilitation.

[135]  D McAlpine,et al.  Responses of neurons in the inferior colliculus to dynamic interaural phase cues: evidence for a mechanism of binaural adaptation. , 2000, Journal of neurophysiology.

[136]  Stephan Getzmann,et al.  Auditory motion perception: onset position and motion direction are encoded in discrete processing stages , 2011, The European journal of neuroscience.

[137]  B. Röder,et al.  Persisting Cross-Modal Changes in Sight-Recovery Individuals Modulate Visual Perception , 2016, Current Biology.

[138]  G. C. Stecker,et al.  Evidence for cue-independent spatial representation in the human auditory cortex during active listening , 2017, Proceedings of the National Academy of Sciences.

[139]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[140]  J. A. Altman,et al.  Are there neurons detecting direction of sound source motion? , 1968, Experimental neurology.

[141]  R. Goebel,et al.  Individual faces elicit distinct response patterns in human anterior temporal cortex , 2007, Proceedings of the National Academy of Sciences.

[142]  S. Lomber,et al.  Double dissociation of 'what' and 'where' processing in auditory cortex , 2008, Nature Neuroscience.

[143]  Robert T. Knight,et al.  Cerebral Responses to Change in Spatial Location of Unattended Sounds , 2007, Neuron.

[144]  Claude Alain,et al.  Assessing the auditory dual-pathway model in humans , 2004, NeuroImage.

[145]  G. Vandewalle,et al.  Functional specialization for auditory–spatial processing in the occipital cortex of congenitally blind humans , 2011, Proceedings of the National Academy of Sciences.

[146]  G. Recanzone,et al.  Effects of stimulus azimuth and intensity on the single-neuron activity in the auditory cortex of the alert macaque monkey. , 2006, Journal of neurophysiology.

[147]  Jeremy Freeman,et al.  Motion Direction Biases and Decoding in Human Visual Cortex , 2014, The Journal of Neuroscience.