The development of multisensory processes

To understand the development of sensory processes, it is necessary not only to look at the maturation of each of the sensory systems in isolation, but also to study the development of the nervous system’s capacity to integrate information across the different senses. It is through such multisensory integration that a coherent perceptual gestalt of the world comes to be generated. In the adult brain, multisensory convergence and integration take place at a number of brainstem and cortical sites, where individual neurons have been found that respond to multisensory stimuli with patterns of activation that depend on the nature of the stimulus complex and the intrinsic properties of the neuron. Parallels between the responses of these neurons and multisensory behavior and perception suggest that they are the substrates that underlie these cognitive processes. In both cat and monkey models, the development of these multisensory neurons and the appearance of their integrative capacity is a gradual postnatal process. For subcortical structures (i.e., the superior colliculus) this maturational process appears to be gated by the appearance of functional projections from regions of association cortex. The slow postnatal maturation of multisensory processes, coupled with its dependency on functional corticotectal connections, suggested that the development of multisensory integration may be tied to sensory experiences acquired during postnatal life. In support of this, eliminating experience in one sensory modality (i.e., vision) during postnatal development severely compromises the integration of multisensory cues. Research is ongoing to better elucidate the critical development antecedents for the emergence of normal multisensory capacity.

[1]  M. Wallace,et al.  Sensory and Multisensory Responses in the Newborn Monkey Superior Colliculus , 2001, The Journal of Neuroscience.

[2]  B. Stein,et al.  The Merging of the Senses , 1993 .

[3]  C. Blakemore,et al.  Developmental plasticity in the visual and auditory representations in the mammalian superior colliculus , 1988, Nature.

[4]  Laurence R. Harris,et al.  Integration of visual and auditory space in the mammalian superior colliculus , 1980, Nature.

[5]  D. Robinson Eye movements evoked by collicular stimulation in the alert monkey. , 1972, Vision research.

[6]  Patrick Haggard,et al.  Persistence of visual–tactile enhancement in humans , 2004, Neuroscience Letters.

[7]  M T Wallace,et al.  Sensory organization of the superior colliculus in cat and monkey. , 1996, Progress in brain research.

[8]  L. Chalupa,et al.  Responses of visual, somatosensory, and auditory neurones in the golden hamster's superior colliculus , 1977, The Journal of physiology.

[9]  S. Shimojo,et al.  Visual illusion induced by sound. , 2002, Brain research. Cognitive brain research.

[10]  R. Andersen,et al.  Intentional maps in posterior parietal cortex. , 2002, Annual review of neuroscience.

[11]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[12]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[13]  M. Wallace,et al.  Representation and integration of multiple sensory inputs in primate superior colliculus. , 1996, Journal of neurophysiology.

[14]  J. K. Harting,et al.  The Mammalian Superior Colliculus: Studies of Its Morphology and Connections , 1984 .

[15]  D. Hubel,et al.  Responses to visual stimulation and relationship between visual, auditory, and somatosensory inputs in mouse superior colliculus. , 1975, Journal of neurophysiology.

[16]  H. Clamann,et al.  The control of eye movements by the superior colliculus in the alert cat , 1976, Brain Research.

[17]  M. Alex Meredith,et al.  Neurons and behavior: the same rules of multisensory integration apply , 1988, Brain Research.

[18]  M A Meredith,et al.  The visuotopic component of the multisensory map in the deep laminae of the cat superior colliculus , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  M. Bornstein,et al.  Development in Infancy , 1982 .

[20]  M. Wallace,et al.  Visual Localization Ability Influences Cross-Modal Bias , 2003, Journal of Cognitive Neuroscience.

[21]  O. D. Creutzfeldt,et al.  Connections of the anterior ectosylvian visual area (AEV) , 2004, Experimental Brain Research.

[22]  B. Stein,et al.  Somatotopic component of the multisensory map in the deep laminae of the cat superior colliculus , 1991, The Journal of comparative neurology.

[23]  W. Jiang,et al.  Two cortical areas mediate multisensory integration in superior colliculus neurons. , 2001, Journal of neurophysiology.

[24]  M. Cynader,et al.  Receptive-field organization of monkey superior colliculus. , 1972, Journal of neurophysiology.

[25]  David J. Lewkowicz,et al.  Development of intersensory perception in human infants. , 1994 .

[26]  J. Gibson The Senses Considered As Perceptual Systems , 1967 .

[27]  T T Norton,et al.  Receptive-field properties of superior colliculus cells and development of visual behavior in kittens. , 1974, Journal of neurophysiology.

[28]  P. Reuter-Lorenz,et al.  Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. , 1994, Journal of experimental psychology. Human perception and performance.

[29]  D. Irvine,et al.  Auditory response properties of neurons in the anterior ectosylvian sulcus of the cat , 1986, Brain Research.

[30]  A R Palmer,et al.  Cells responsive to free‐field auditory stimuli in guinea‐pig superior colliculus: distribution and response properties. , 1983, The Journal of physiology.

[31]  E. Knudsen,et al.  Creating a unified representation of visual and auditory space in the brain. , 1995, Annual review of neuroscience.

[32]  J. C. Middlebrooks,et al.  A neural code for auditory space in the cat's superior colliculus , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  R. Andersen,et al.  Models of the Posterior Parietal Cortex Which Perform Multimodal Integration and Represent Space in Several Coordinate Frames , 2000, Journal of Cognitive Neuroscience.

[34]  James K. Kroger,et al.  Cross-modal and cross-temporal association in neurons of frontal cortex , 2000, Nature.

[35]  B. Stein,et al.  Spatial factors determine the activity of multisensory neurons in cat superior colliculus , 1986, Brain Research.

[36]  D. Jassik-Gerschenfeld,et al.  Somesthetic and Visual Responses of Superior Colliculus Neurones , 1965, Nature.

[37]  H. R. Clemo,et al.  Somatosensory cortex: a ‘new’ somatotopic representation , 1982, Brain Research.

[38]  L. Mucke,et al.  Physiologic and anatomic investigation of a visual cortical area situated in the ventral bank of the anterior ectosylvian sulcus of the cat , 2004, Experimental Brain Research.

[39]  Jeffery A Winer,et al.  A multisensory zone in rat parietotemporal cortex: Intra‐ and extracellular physiology and thalamocortical connections , 2003, The Journal of comparative neurology.

[40]  Barry E Stein,et al.  Neuron-specific response characteristics predict the magnitude of multisensory integration. , 2003, Journal of neurophysiology.

[41]  M HERSHENSON,et al.  Reaction time as a measure of intersensory facilitation. , 1962, Journal of experimental psychology.

[42]  R. Wurtz,et al.  Activity of superior colliculus in behaving monkey. I. Visual receptive fields of single neurons. , 1972, Journal of neurophysiology.

[43]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.

[44]  C. Spence,et al.  Multimodal visual–somatosensory integration in saccade generation , 2003, Neuropsychologia.

[45]  M. Wallace,et al.  Superior colliculus lesions preferentially disrupt multisensory orientation , 2004, Neuroscience.

[46]  B E Stein,et al.  Corticotectal and corticothalamic efferent projections of SIV somatosensory cortex in cat. , 1983, Journal of neurophysiology.

[47]  G. Schneider,et al.  Topography of visual and somatosensory projections to the superior colliculus of the golden hamster , 1978, Brain Research.

[48]  B E Stein,et al.  Sequence of changes in properties of neurons of superior colliculus of the kitten during maturation. , 1973, Journal of neurophysiology.

[49]  B. Stein,et al.  Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. , 1986, Journal of neurophysiology.

[50]  D. Hubel,et al.  Early Exploration of the Visual Cortex , 1998, Neuron.

[51]  M. T. Wallace,et al.  Visual, auditory and somatosensory convergence in output neurons of the cat superior colliculus: multisensory properties of the tecto-reticulo-spinal projection , 2005, Experimental Brain Research.

[52]  Barbara G. Wickelgren,et al.  Superior Colliculus: Some Receptive Field Properties of Bimodally Responsive Cells , 1971, Science.

[53]  A GRAYBIEL,et al.  Oculogravic illusion. , 1952, A.M.A. archives of ophthalmology.

[54]  C. K. Peck,et al.  Spatial disparity affects visual-auditory interactions in human sensorimotor processing , 1998, Experimental Brain Research.

[55]  G. Rizzolatti,et al.  Coding of peripersonal space in inferior premotor cortex (area F4). , 1996, Journal of neurophysiology.

[56]  M T Wallace,et al.  Onset of cross-modal synthesis in the neonatal superior colliculus is gated by the development of cortical influences. , 2000, Journal of neurophysiology.

[57]  R. Sekuler,et al.  Sound alters visual motion perception , 1997, Nature.

[58]  Heinz Werner,et al.  Comparative Psychology of Mental Development , 1942 .

[59]  M. Wallace,et al.  A revised view of sensory cortical parcellation , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[60]  Josef P. Rauschecker,et al.  The Development of Intersensory Perception: Comparative Perspectives , 1996, Journal of Cognitive Neuroscience.

[61]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[62]  Christopher T. Lovelace,et al.  An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. , 2003, Brain research. Cognitive brain research.

[63]  M. Frens,et al.  Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements , 1995, Perception & psychophysics.

[64]  H. Birch,et al.  INTERSENSORY DEVELOPMENT IN CHILDREN. , 1963, Monographs of the Society for Research in Child Development.

[65]  Steiner Be,et al.  Control of pinna movements and sensorimotor register in cat superior colliculus. , 1981 .

[66]  P. L. Adams THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[67]  M T Wallace,et al.  Cross-modal synthesis in the midbrain depends on input from cortex. , 1994, Journal of neurophysiology.

[68]  V. Jousmäki,et al.  Parchment-skin illusion: sound-biased touch , 1998, Current Biology.

[69]  Barry E. Stein,et al.  Eye movements evoked by electrical stimulation in the superior colliculus of rats and hamsters , 1982, Brain Research.

[70]  E I Knudsen,et al.  Visual instruction of the neural map of auditory space in the developing optic tectum. , 1991, Science.

[71]  M. Wallace,et al.  Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. , 1993, Journal of neurophysiology.

[72]  M A Meredith,et al.  Functional development of a central visual map in cat. , 1994, Journal of neurophysiology.

[73]  C. Olson,et al.  Ectosylvian visual area of the cat: Location, retinotopic organization, and connections , 1987, The Journal of comparative neurology.

[74]  B. Stein,et al.  Properties of superior colliculus neurons in the golden hamster , 1979, The Journal of comparative neurology.

[75]  M T Wallace,et al.  Development of Multisensory Neurons and Multisensory Integration in Cat Superior Colliculus , 1997, The Journal of Neuroscience.

[76]  B. Gordon,et al.  Receptive fields in deep layers of cat superior colliculus. , 1973, Journal of neurophysiology.

[77]  B E Stein,et al.  Control of pinna movements and sensorimotor register in cat superior colliculus. , 1981, Brain, behavior and evolution.

[78]  J. Andreassi,et al.  Effects of bisensory stimulation on reaction time and the evoked cortical potential , 1975 .

[79]  Franco Lepore,et al.  Sensory modality distribution in the anterior ectosylvian cortex (AEC) of cats , 2004, Experimental Brain Research.

[80]  Colin Blakemore,et al.  Functional organization in the superior colliculus of the golden hamster , 1976, The Journal of comparative neurology.

[81]  H. Birch,et al.  Visual differentiation, intersensory integration, and voluntary motor control. , 1967, Monographs of the Society for Research in Child Development.

[82]  P. Schiller,et al.  Single-unit recording and stimulation in superior colliculus of the alert rhesus monkey. , 1972, Journal of neurophysiology.

[83]  C. Gross,et al.  Visuospatial properties of ventral premotor cortex. , 1997, Journal of neurophysiology.

[84]  M. Wallace,et al.  Integration of multiple sensory modalities in cat cortex , 2004, Experimental Brain Research.

[85]  N. Bolognini,et al.  Enhancement of visual perception by crossmodal visuo-auditory interaction , 2002, Experimental Brain Research.

[86]  Cristiana Cavina-Pratesi,et al.  Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time , 2002, Experimental Brain Research.

[87]  D L Sparks,et al.  Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. , 1986, Physiological reviews.

[88]  Hans Colonius,et al.  Visual-tactile spatial interaction in saccade generation , 2003, Experimental Brain Research.

[89]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[90]  A. J. King,et al.  Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus , 2004, Experimental Brain Research.

[91]  A. R. Palmer,et al.  A monaural space map in the guinea-pig superior colliculus , 1985, Hearing Research.

[92]  H. R. Clemo,et al.  Auditory cortical projection from the anterior ectosylvian sulcus (Field AES) to the superior colliculus in the cat: an anatomical and electrophysiological study. , 1989, The Journal of comparative neurology.

[93]  B. Stein,et al.  Two Corticotectal Areas Facilitate Multisensory Orientation Behavior , 2002, Journal of Cognitive Neuroscience.

[94]  A J Van Opstal,et al.  Auditory-visual interactions subserving goal-directed saccades in a complex scene. , 2002, Journal of neurophysiology.

[95]  T H MEIKLE,et al.  THE ROLE OF THE SUPERIOR COLLICULUS IN VISUALLY GUIDED BEHAVIOR. , 1965, Experimental neurology.

[96]  B E Stein,et al.  Unimodal and multimodal response properties of neurons in the cat's superior colliculus. , 1972, Experimental neurology.

[97]  H. R. Clemo,et al.  Organization of a fourth somatosensory area of cortex in cat. , 1983, Journal of neurophysiology.

[98]  M. Wallace,et al.  Multisensory integration in the superior colliculus of the alert cat. , 1998, Journal of neurophysiology.

[99]  B. Stein,et al.  Interactions among converging sensory inputs in the superior colliculus. , 1983, Science.

[100]  H Colonius,et al.  A two-stage model for visual-auditory interaction in saccadic latencies , 2001, Perception & psychophysics.

[101]  Ankoor S. Shah,et al.  Auditory Cortical Neurons Respond to Somatosensory Stimulation , 2003, The Journal of Neuroscience.

[102]  B E Stein,et al.  Relationship between visual and tactile representations in cat superior colliculus. , 1976, Journal of neurophysiology.

[103]  I. Howard,et al.  Human Spatial Orientation , 1966 .

[104]  F. Reinoso-suárez,et al.  Projections from non-visual cortical areas to the superior colliculus demonstrated by retrograde transport of HRP in the cat , 1980, Brain Research.

[105]  B. Stein,et al.  Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[106]  Lawrence G. McDade,et al.  Behavioral Indices of Multisensory Integration: Orientation to Visual Cues is Affected by Auditory Stimuli , 1989, Journal of Cognitive Neuroscience.

[107]  Franco Lepore,et al.  Sensory interactions in the anterior ectosylvian cortex of cats , 2004, Experimental Brain Research.

[108]  B E Stein,et al.  Nonequivalent visual, auditory, and somatic corticotectal influences in cat. , 1978, Journal of neurophysiology.