Auditory and Visual Modulation of Temporal Lobe Neurons in Voice-Sensitive and Association Cortices

Effective interactions between conspecific individuals can depend upon the receiver forming a coherent multisensory representation of communication signals, such as merging voice and face content. Neuroimaging studies have identified face- or voice-sensitive areas (Belin et al., 2000; Petkov et al., 2008; Tsao et al., 2008), some of which have been proposed as candidate regions for face and voice integration (von Kriegstein et al., 2005). However, it was unclear how multisensory influences occur at the neuronal level within voice- or face-sensitive regions, especially compared with classically defined multisensory regions in temporal association cortex (Stein and Stanford, 2008). Here, we characterize auditory (voice) and visual (face) influences on neuronal responses in a right-hemisphere voice-sensitive region in the anterior supratemporal plane (STP) of Rhesus macaques. These results were compared with those in the neighboring superior temporal sulcus (STS). Within the STP, our results show auditory sensitivity to several vocal features, which was not evident in STS units. We also newly identify a functionally distinct neuronal subpopulation in the STP that appears to carry the area's sensitivity to voice identity related features. Audiovisual interactions were prominent in both the STP and STS. However, visual influences modulated the responses of STS neurons with greater specificity and were more often associated with congruent voice-face stimulus pairings than STP neurons. Together, the results reveal the neuronal processes subserving voice-sensitive fMRI activity patterns in primates, generate hypotheses for testing in the visual modality, and clarify the position of voice-sensitive areas within the unisensory and multisensory processing hierarchies.

[1]  U. Noppeney,et al.  Distinct Functional Contributions of Primary Sensory and Association Areas to Audiovisual Integration in Object Categorization , 2010, The Journal of Neuroscience.

[2]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[3]  J. Fritz,et al.  Rhesus macaques spontaneously perceive formants in conspecific vocalizations. , 2006, The Journal of the Acoustical Society of America.

[4]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[5]  M. Mishkin,et al.  Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex , 1999, Nature Neuroscience.

[6]  Alfred Anwander,et al.  Direct Structural Connections between Voice- and Face-Recognition Areas , 2011, The Journal of Neuroscience.

[7]  M Hallett,et al.  Intra- and interhemispheric connections of the neocortical auditory system in the rhesus monkey. , 1969, Brain research.

[8]  R. Zatorre,et al.  Voice-selective areas in human auditory cortex , 2000, Nature.

[9]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.

[10]  Andreas Kleinschmidt,et al.  Interaction of Face and Voice Areas during Speaker Recognition , 2005, Journal of Cognitive Neuroscience.

[11]  N. Logothetis,et al.  Visual modulation of neurons in auditory cortex. , 2008, Cerebral cortex.

[12]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[13]  Roy D. Patterson,et al.  Vocal-Tract Resonances as Indexical Cues in Rhesus Monkeys , 2007, Current Biology.

[14]  Joost X. Maier,et al.  Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex , 2005 .

[15]  Christoph Kayser,et al.  Do early sensory cortices integrate cross-modal information? , 2007, Brain Structure and Function.

[16]  A. Kleinschmidt,et al.  Modulation of neural responses to speech by directing attention to voices or verbal content. , 2003, Brain research. Cognitive brain research.

[17]  Dylan M. Jones,et al.  Intra- and inter-modal repetition priming of familiar faces and voices. , 1997, British journal of psychology.

[18]  T. Stanford,et al.  Evaluating the Operations Underlying Multisensory Integration in the Cat Superior Colliculus , 2005, The Journal of Neuroscience.

[19]  J. Kaas,et al.  Subdivisions of AuditoryCortex and Levels of Processing in Primates , 1998, Audiology and Neurotology.

[20]  John H. R. Maunsell,et al.  Visual processing in monkey extrastriate cortex. , 1987, Annual review of neuroscience.

[21]  John J. Foxe,et al.  Multisensory contributions to low-level, ‘unisensory’ processing , 2005, Current Opinion in Neurobiology.

[22]  Bruno B Averbeck,et al.  Neural representation of vocalizations in the primate ventrolateral prefrontal cortex. , 2005, Journal of neurophysiology.

[23]  J. Duhamel,et al.  Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey , 2007, The Journal of Neuroscience.

[24]  L. Benevento,et al.  Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey , 1977, Experimental Neurology.

[25]  J. Sergent,et al.  Functional neuroanatomy of face and object processing. A positron emission tomography study. , 1992, Brain : a journal of neurology.

[26]  Frank Schneider,et al.  Incongruence effects in crossmodal emotional integration , 2011, NeuroImage.

[27]  R. Andersen,et al.  Responses to auditory stimuli in macaque lateral intraparietal area. II. Behavioral modulation. , 1999, Journal of neurophysiology.

[28]  M. Mishkin,et al.  Species-specific calls evoke asymmetric activity in the monkey's temporal poles , 2004, Nature.

[29]  D. Pandya,et al.  Parietal, temporal, and occipita projections to cortex of the superior temporal sulcus in the rhesus monkey: A retrograde tracer study , 1994, The Journal of comparative neurology.

[30]  Anne-Lise Giraud,et al.  Distinct functional substrates along the right superior temporal sulcus for the processing of voices , 2004, NeuroImage.

[31]  Yukiko Kikuchi,et al.  Hierarchical Auditory Processing Directed Rostrally along the Monkey's Supratemporal Plane , 2010, The Journal of Neuroscience.

[32]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[33]  G. Sáry,et al.  Auditory modulation of the inferior temporal cortex neurons in rhesus monkey. , 2011, Physiological research.

[34]  James K. Kroger,et al.  Cross-modal and cross-temporal association in neurons of frontal cortex , 2000, Nature.

[35]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[36]  S. Campanella,et al.  Integrating face and voice in person perception , 2007, Trends in Cognitive Sciences.

[37]  A M Galaburda,et al.  The intrinsic architectonic and connectional organization of the superior temporal region of the rhesus monkey , 1983, The Journal of comparative neurology.

[38]  J. Kaas,et al.  Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys , 1998, The Journal of comparative neurology.

[39]  Christoph Kayser,et al.  Modulation of Visual Responses in the Superior Temporal Sulcus by Audio-Visual Congruency , 2010, Front. Integr. Neurosci..

[40]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[41]  J. Kaas,et al.  Subdivisions of auditory cortex and processing streams in primates. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[42]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[43]  I. Nelken,et al.  Physiological and Anatomical Evidence for Multisensory Interactions in Auditory Cortex , 2006, Cerebral cortex.

[44]  N. Mesgarani,et al.  Selective cortical representation of attended speaker in multi-talker speech perception , 2012, Nature.

[45]  P. Goldman-Rakic,et al.  Auditory belt and parabelt projections to the prefrontal cortex in the Rhesus monkey , 1999, The Journal of comparative neurology.

[46]  Bruno B Averbeck,et al.  Integration of Auditory and Visual Communication Information in the Primate Ventrolateral Prefrontal Cortex , 2006, The Journal of Neuroscience.

[47]  P. B. Cipolloni,et al.  Connectional analysis of the ipsilateral and contralateral afferent neurons of the superior temporal region in the rhesus monkey , 1989, The Journal of comparative neurology.

[48]  M. Mishkin,et al.  Serial and parallel processing in rhesus monkey auditory cortex , 1997, The Journal of comparative neurology.

[49]  Christoph Kayser,et al.  Voice Cells in the Primate Temporal Lobe , 2011, Current Biology.

[50]  Christoph Kayser,et al.  Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex , 2007, The Journal of Neuroscience.

[51]  N. Logothetis,et al.  A voice region in the monkey brain , 2008, Nature Neuroscience.

[52]  N. Logothetis,et al.  Functional Imaging Reveals Numerous Fields in the Monkey Auditory Cortex , 2006, PLoS biology.

[53]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[54]  Christoph Kayser,et al.  Spatial Organization of Multisensory Responses in Temporal Association Cortex , 2009, The Journal of Neuroscience.

[55]  T. Hackett,et al.  Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[56]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[57]  C. Schroeder,et al.  The Leading Sense: Supramodal Control of Neurophysiological Context by Attention , 2009, Neuron.

[58]  Jude F. Mitchell,et al.  Differential Attention-Dependent Response Modulation across Cell Classes in Macaque Visual Area V4 , 2007, Neuron.

[59]  Lizabeth M Romanski,et al.  Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. , 2007, Cerebral cortex.

[60]  Dora E Angelaki,et al.  Functional Specializations of the Ventral Intraparietal Area for Multisensory Heading Discrimination , 2013, The Journal of Neuroscience.

[61]  R. Zatorre,et al.  Human temporal-lobe response to vocal sounds. , 2002, Brain research. Cognitive brain research.

[62]  C. Schroeder,et al.  Neuronal Oscillations and Multisensory Interaction in Primary Auditory Cortex , 2007, Neuron.

[63]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[64]  R. Zatorre,et al.  Adaptation to speaker's voice in right anterior temporal lobe , 2003, Neuroreport.

[65]  U. Noppeney,et al.  Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. , 2010, Cerebral cortex.

[66]  R. Andersen,et al.  Responses to auditory stimuli in macaque lateral intraparietal area. I. Effects of training. , 1999, Journal of neurophysiology.

[67]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[68]  Chris I. Baker,et al.  Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions , 2005, Journal of Cognitive Neuroscience.

[69]  P S Goldman-Rakic,et al.  Topographic organization of medial pulvinar connections with the prefrontal cortex in the rhesus monkey , 1997, The Journal of comparative neurology.