Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey

While multisensory integration is thought to occur in higher hierarchical cortical areas, recent studies in man and monkey have revealed plurisensory modulations of activity in areas previously thought to be unimodal. To determine the cortical network involved in multisensory interactions, we performed multiple injections of different retrograde tracers in unimodal auditory (core), somatosensory (1/3b) and visual (V2 and MT) cortical areas of the marmoset. We found three types of heteromodal connections linking unimodal sensory areas. Visuo‐somatosensory projections were observed originating from visual areas [probably the ventral and dorsal fundus of the superior temporal area (FSTv and FSTd), and middle temporal crescent (MTc)] toward areas 1/3b. Somatosensory projections to the auditory cortex were present from S2 and the anterior bank of the lateral sulcus. Finally, a visuo‐auditory projection arises from an area anterior to the superior temporal sulcus (STS) toward the auditory core. Injections in different sensory regions allow us to define the frontal convexity and the temporal opercular caudal cortex as putative polysensory areas. A quantitative analysis of the laminar distribution of projecting neurons showed that heteromodal connections could be either feedback or feedforward. Taken together, our results provide the anatomical pathway for multisensory integration at low levels of information processing in the primate and argue against a strict hierarchical model.

[1]  Henning Scheich,et al.  NON-ACOUSTIC INFLUENCE ON NEURAL ACTIVITY IN AUDITORY CORTEX , 2005 .

[2]  Dylan F. Cooke,et al.  Sensorimotor integration in the precentral gyrus: polysensory neurons and defensive movements. , 2004, Journal of neurophysiology.

[3]  P. Barone,et al.  Postnatal development of alkaline phosphatase activity correlates with the maturation of neurotransmission in the cerebral cortex , 2005, The Journal of comparative neurology.

[4]  H. Burton,et al.  Cortical activity to vibrotactile stimulation: An fMRI study in blind and sighted individuals , 2004, Human brain mapping.

[5]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[6]  J. Alonso,et al.  Lateral-posterior and pulvinar reaching cells—comparison with parietal area 5a: a study in behaving Macaca nemestrina monkeys , 2004, Experimental Brain Research.

[7]  Leah Krubitzer,et al.  Cortical connections of the second somatosensory area and the parietal ventral area in macaque monkeys , 2003, The Journal of comparative neurology.

[8]  F. Rösler,et al.  Speech processing activates visual cortex in congenitally blind humans , 2002, The European journal of neuroscience.

[9]  D. Bavelier,et al.  Cross-modal plasticity: where and how? , 2002, Nature Reviews Neuroscience.

[10]  L. Aitkin,et al.  Connections of the primary auditory cortex in the common marmoset, Callithrix jacchus jacchus , 1988, The Journal of comparative neurology.

[11]  J. Kaas,et al.  Connections of primary auditory cortex in the new world monkey, Saguinus , 1989, The Journal of comparative neurology.

[12]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  Alvaro Pascual-Leone,et al.  Behavioral and neuroplastic changes in the blind: evidence for functionally relevant cross-modal interactions , 2004, Journal of Physiology-Paris.

[14]  S. Hillyard,et al.  Involuntary orienting to sound improves visual perception , 2000, Nature.

[15]  D. I. Perrett,et al.  Visual and somatosensory processing in the macaque temporal cortex: the role of ‘expectation’ , 2004, Experimental Brain Research.

[16]  H. Barbas,et al.  Cortical structure predicts the pattern of corticocortical connections. , 1997, Cerebral cortex.

[17]  P. Barone,et al.  Areal and subcellular localization of the ubiquitous alkaline phosphatase in the primate cerebral cortex: evidence for a role in neurotransmission. , 2004, Cerebral cortex.

[18]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[19]  F. Mauguière,et al.  Pulvinar-latero posterior afferents to cortical area 7 in monkeys demonstrated by horseradish peroxidase tracing technique , 1977, Experimental Brain Research.

[20]  J. Kaas,et al.  Architectonic identification of the core region in auditory cortex of macaques, chimpanzees, and humans , 2001, The Journal of comparative neurology.

[21]  François Klam,et al.  ã Federation of European Neuroscience Societies Visual±vestibular interactive responses in the macaque ventral intraparietal area (VIP) , 2022 .

[22]  John J. Foxe,et al.  The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. , 2002, Brain research. Cognitive brain research.

[23]  Tim Rons Novel sensations in the congenitally blind , 1996, Nature.

[24]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[25]  Keiji Tanaka,et al.  Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. , 1988, Journal of neurophysiology.

[26]  K. Alho,et al.  Auditory and somatosensory event-related brain potentials in early blind humans , 2004, Experimental Brain Research.

[27]  Scott T. Grafton,et al.  Feeling with the mind's eye , 1997, Neuroreport.

[28]  John J. Foxe,et al.  The case for feedforward multisensory convergence during early cortical processing , 2005, Neuroreport.

[29]  John J. Foxe,et al.  Multisensory Convergence in Early Cortical Processing. , 2004 .

[30]  D. V. van Essen,et al.  Mapping of architectonic subdivisions in the macaque monkey, with emphasis on parieto‐occipital cortex , 2000, The Journal of comparative neurology.

[31]  G. Orban,et al.  Visual Motion Processing Investigated Using Contrast Agent-Enhanced fMRI in Awake Behaving Monkeys , 2001, Neuron.

[32]  L A Krubitzer,et al.  Cortical connections of MT in four species of primates: Areal, modular, and retinotopic patterns , 1990, Visual Neuroscience.

[33]  Christopher T. Lovelace,et al.  An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. , 2003, Brain research. Cognitive brain research.

[34]  Jon H Kaas,et al.  Anatomical and functional organization of somatosensory areas of the lateral fissure of the New World titi monkey (Callicebus moloch) , 2004, The Journal of comparative neurology.

[35]  E. G. Jones,et al.  Intracortical connectivity of architectonic fields in the somatic sensory, motor and parietal cortex of monkeys , 1978, The Journal of comparative neurology.

[36]  J. H. Kaas,et al.  Ablations of areas 3a and 3b of monkey somatosensory cortex abolish cutaneous responsivity in area 1 , 1990, Brain Research.

[37]  L Krubitzer,et al.  Thalamo‐cortical connections of areas 3a and M1 in marmoset monkeys , 2001, The Journal of comparative neurology.

[38]  C. Spence,et al.  Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism , 2000, Neuroreport.

[39]  H. Kennedy,et al.  Laminar Distribution of Neurons in Extrastriate Areas Projecting to Visual Areas V1 and V4 Correlates with the Hierarchical Rank and Indicates the Operation of a Distance Rule , 2000, The Journal of Neuroscience.

[40]  C. Spence,et al.  The Handbook of Multisensory Processing , 2004 .

[41]  B. Stein Neural mechanisms for synthesizing sensory information and producing adaptive behaviors , 1998, Experimental Brain Research.

[42]  K Sathian,et al.  Visual cortical activity during tactile perception in the sighted and the visually deprived. , 2005, Developmental psychobiology.

[43]  M S Graziano,et al.  A System of Multimodal Areas in the Primate Brain , 2001, Neuron.

[44]  J. Downar,et al.  A multimodal cortical network for the detection of changes in the sensory environment , 2000, Nature Neuroscience.

[45]  John J. Foxe,et al.  Multisensory contributions to low-level, ‘unisensory’ processing , 2005, Current Opinion in Neurobiology.

[46]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.

[47]  J Bullier,et al.  Developmental remodeling of primate visual cortical pathways. , 1995, Cerebral cortex.

[48]  B. Stein,et al.  Interactions among converging sensory inputs in the superior colliculus. , 1983, Science.

[49]  H. Kennedy,et al.  Anatomical Evidence of Multimodal Integration in Primate Striate Cortex , 2002, The Journal of Neuroscience.

[50]  J. Kaas,et al.  Subdivisions and connections of auditory cortex in owl monkeys , 1992, The Journal of comparative neurology.

[51]  J. Kaas,et al.  Connectional and Architectonic Evidence for Dorsal and Ventral V3, and Dorsomedial Area in Marmoset Monkeys , 2001, The Journal of Neuroscience.

[52]  K. Rockland,et al.  Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey , 1979, Brain Research.

[53]  Lisa A. de la Mothe,et al.  A comparison of neuron response properties in areas A1 and CM of the marmoset monkey auditory cortex: tones and broadband noise. , 2005, Journal of neurophysiology.

[54]  C. Spence,et al.  Audiotactile interactions in roughness perception , 2002, Experimental Brain Research.

[55]  L A Krubitzer,et al.  The somatosensory thalamus of monkeys: Cortical connections and a redefinition of nuclei in marmosets , 1992, The Journal of comparative neurology.

[56]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[57]  M. A. Frens,et al.  A quantitative study of auditory-evoked saccadic eye movements in two dimensions , 2004, Experimental Brain Research.

[58]  J. Kaas,et al.  Tonotopic organization, architectonic fields, and connections of auditory cortex in macaque monkeys , 1993, The Journal of comparative neurology.

[59]  P. Goldman-Rakic,et al.  Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections , 1989, The Journal of comparative neurology.

[60]  M. Hallett,et al.  Activation of the primary visual cortex by Braille reading in blind subjects , 1996, Nature.

[61]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[62]  J. Kaas,et al.  Topographic patterns of V2 cortical connections in macaque monkeys , 1996, The Journal of comparative neurology.

[63]  M. Rosa Visuotopic Organization of Primate Extrastriate Cortex , 1997 .

[64]  Ione Fine,et al.  Visual stimuli activate auditory cortex in the deaf , 2001, Nature Neuroscience.

[65]  C. Spence,et al.  What role does multisensory integration play in the visuotactile perception of texture? , 2003, International Journal of Psychophysiology.

[66]  Brigitte Röder,et al.  Compensatory Plasticity as a Consequence of Sensory Loss. , 2004 .

[67]  E. Rolls,et al.  Functional subdivisions of the temporal lobe neocortex , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[68]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[69]  P. H. Schiller,et al.  State dependent activity in monkey visual cortex , 2004, Experimental Brain Research.

[70]  F. Gallyas Silver staining of myelin by means of physical development. , 1979, Neurological research.

[71]  H. Barbas Pattern in the laminar origin of corticocortical connections , 1986, The Journal of comparative neurology.

[72]  Leslie G. Ungerleider,et al.  Pathways for motion analysis: Cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque , 1990, The Journal of comparative neurology.

[73]  J. Pernier,et al.  Dynamics of cortico-subcortical cross-modal operations involved in audio-visual object detection in humans. , 2002, Cerebral cortex.

[74]  A. Morel,et al.  Connections of visual areas of the upper temporal lobe of owl monkeys: the MT crescent and dorsal and ventral subdivisions of FST , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[75]  H. Scheich,et al.  Nonauditory Events of a Behavioral Procedure Activate Auditory Cortex of Highly Trained Monkeys , 2005, The Journal of Neuroscience.

[76]  Leslie G. Ungerleider,et al.  Multiple visual areas in the caudal superior temporal sulcus of the macaque , 1986, The Journal of comparative neurology.

[77]  André Dufour,et al.  Importance of attentional mechanisms in audiovisual links , 1999, Experimental Brain Research.

[78]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[79]  M. Merzenich,et al.  Frequency representation in auditory cortex of the common marmoset (Callithrix jacchus jacchus) , 1986, The Journal of comparative neurology.

[80]  Leslie G. Ungerleider,et al.  Cortical connections of visual area MT in the macaque , 1986, The Journal of comparative neurology.

[81]  M. Rosa,et al.  Visual areas in lateral and ventral extrastriate cortices of the marmoset monkey , 2000, The Journal of comparative neurology.

[82]  M. Wallace,et al.  Integration of multiple sensory modalities in cat cortex , 2004, Experimental Brain Research.

[83]  A. John Van Opstal,et al.  Plasticity in human sound localization induced by compressed spatial vision , 2003, Nature Neuroscience.

[84]  G. Elston,et al.  Visuotopic organisation and neuronal response selectivity for direction of motion in visual areas of the caudal temporal lobe of the marmoset monkey (Callithrix jacchus): Middle temporal area, middle temporal crescent, and surrounding cortex , 1998, The Journal of comparative neurology.

[85]  M P Young,et al.  Indeterminate Organization of the Visual System , 1996, Science.

[86]  Leah Krubitzer,et al.  The organization and connections of anterior and posterior parietal cortex in titi monkeys: do New World monkeys have an area 2? , 2005, Cerebral cortex.

[87]  J. Kaas,et al.  Thalamocortical connections of the parabelt auditory cortex in macaque monkeys , 1998, The Journal of comparative neurology.

[88]  David C Lyon,et al.  Cortical and thalamic connections of the parietal ventral somatosensory area in marmoset monkeys (Callithrix jacchus) , 2002, The Journal of comparative neurology.

[89]  J. Kaas Theories of Visual Cortex Organization in Primates , 1997 .

[90]  J. Kaas,et al.  Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys , 1998, The Journal of comparative neurology.

[91]  S. Zeki,et al.  The Organization of Connections between Areas V5 and V1 in Macaque Monkey Visual Cortex , 1989, The European journal of neuroscience.

[92]  J. Bullier,et al.  Topography of visual cortex connections with frontal eye field in macaque: convergence and segregation of processing streams , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[93]  M. Gamberini,et al.  Resolving the organization of the New World monkey third visual complex: The dorsal extrastriate cortex of the marmoset (Callithrix jacchus) , 2005, The Journal of comparative neurology.

[94]  Y. Yen,et al.  Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli , 2002, Journal of Cognitive Neuroscience.

[95]  D. V. van Essen,et al.  Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey , 2000, The Journal of comparative neurology.

[96]  C. Darian‐Smith,et al.  Ipsilateral cortical projections to areas 3a, 3b, and 4 in the macaque monkey , 1993, The Journal of comparative neurology.

[97]  John H. R. Maunsell,et al.  The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[98]  H. Barbas,et al.  The laminar pattern of connections between prefrontal and anterior temporal cortices in the Rhesus monkey is related to cortical structure and function. , 2000, Cerebral cortex.

[99]  Reinhard Konig and The auditory cortex : a synthesis of human and animal research , 2005 .

[100]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[101]  C. Cusick,et al.  The Superior Temporal Polysensory Region in Monkeys , 1997 .

[102]  R. Desimone,et al.  Visual areas in the temporal cortex of the macaque , 1979, Brain Research.

[103]  L Krubitzer,et al.  Area 3a: topographic organization and cortical connections in marmoset monkeys. , 2001, Cerebral cortex.

[104]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[105]  Dylan F. Cooke,et al.  Defensive movements evoked by air puff in monkeys. , 2003, Journal of neurophysiology.

[106]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[107]  T. Hackett,et al.  Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[108]  M. Wong-Riley Changes in the visual system of monocularly sutured or enucleated cats demonstrable with cytochrome oxidase histochemistry , 1979, Brain Research.

[109]  Michael C. Dorris,et al.  Combined eye-head gaze shifts to visual and auditory targets in humans , 1996, Experimental Brain Research.

[110]  Ryota Kanai,et al.  Changing pitch induced visual motion illusion , 2004, Current Biology.

[111]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[112]  P A Salin,et al.  Corticocortical connections in the visual system: structure and function. , 1995, Physiological reviews.

[113]  B. Stein,et al.  Enhancement of Perceived Visual Intensity by Auditory Stimuli: A Psychophysical Analysis , 1996, Journal of Cognitive Neuroscience.

[114]  P. Belin,et al.  The Auditory Cortex: a Synthesis of Human and Animal Research , 2005 .

[115]  R. Born,et al.  Specificity of Projections from Wide-Field and Local Motion-Processing Regions within the Middle Temporal Visual Area of the Owl Monkey , 2000, The Journal of Neuroscience.

[116]  K. Zilles,et al.  Polymodal Motion Processing in Posterior Parietal and Premotor Cortex A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys , 2001, Neuron.

[117]  Gregory Hickok,et al.  Visual stimuli activate auditory cortex in deaf subjects: evidence from MEG , 2003, Neuroreport.

[118]  Michael J Brammer,et al.  Crossmodal identification , 1998, Trends in Cognitive Sciences.

[119]  J. Kaas,et al.  Studies on the evolution of multiple somatosensory representations in primates: The organization of anterior parietal cortex in the new world callitrichid, Saguinus , 1986, The Journal of comparative neurology.

[120]  Claus C. Hilgetag,et al.  Uniformity and specificity of long-range corticocortical connections in the visual cortex of the cat , 2001, Neurocomputing.

[121]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[122]  H. Burton Visual Cortex Activity in Early and Late Blind People , 2003, The Journal of Neuroscience.

[123]  D J Felleman,et al.  Somatotopic organization of the lateral sulcus of owl monkeys: Area 3b, s‐II, and a ventral somatosensory area , 1989, The Journal of comparative neurology.

[124]  C. Colby,et al.  Heterogeneity of extrastriate visual areas and multiple parietal areas in the Macaque monkey , 1991, Neuropsychologia.

[125]  C. Galletti,et al.  The cortical connections of area V6: an occipito‐parietal network processing visual information , 2001, The European journal of neuroscience.

[126]  C. Spence,et al.  Exogenous spatial cuing studies of human crossmodal attention and multisensory integration , 2004 .

[127]  L A Krubitzer,et al.  The organization and connections of somatosensory cortex in marmosets , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[128]  John J. Foxe,et al.  Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. , 2000, Brain research. Cognitive brain research.

[129]  R Caminiti,et al.  The callosal system of the superior parietal lobule in the monkey , 1985, The Journal of comparative neurology.

[130]  J. Rauschecker,et al.  A Positron Emission Tomographic Study of Auditory Localization in the Congenitally Blind , 2000, The Journal of Neuroscience.

[131]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[132]  H. Kennedy,et al.  Area‐specific laminar distribution of cortical feedback neurons projecting to cat area 17: Quantitative analysis in the adult and during ontogeny , 1998, The Journal of comparative neurology.

[133]  Ankoor S. Shah,et al.  Auditory Cortical Neurons Respond to Somatosensory Stimulation , 2003, The Journal of Neuroscience.