Effect of auditory input on activations in infant diverse cortical regions during audiovisual processing

A fundamental question with regard to perceptual development is how multisensory information is processed in the brain during the early stages of development. Although a growing body of evidence has shown the early emergence of modality‐specific functional differentiation of the cortical regions, the interplay between sensory inputs from different modalities in the developing brain is not well understood. To study the effects of auditory input during audio‐visual processing in 3‐month‐old infants, we evaluated the spatiotemporal cortical hemodynamic responses of 50 infants while they perceived visual objects with or without accompanying sounds. The responses were measured using 94‐channel near‐infrared spectroscopy over the occipital, temporal, and frontal cortices. The effects of sound manipulation were pervasive throughout the diverse cortical regions and were specific to each cortical region. Visual stimuli co‐occurring with sound induced the early‐onset activation of the early auditory region, followed by activation of the other regions. Removal of the sound stimulus resulted in focal deactivation in the auditory regions and reduced activation in the early visual region, the association region of the temporal and parietal cortices, and the anterior prefrontal regions, suggesting multisensory interplay. In contrast, equivalent activations were observed in the lateral occipital and lateral prefrontal regions, regardless of sound manipulation. Our findings indicate that auditory input did not generally enhance overall activation in relation to visual perception, but rather induced specific changes in each cortical region. The present study implies that 3‐month‐old infants may perceive audio‐visual multisensory inputs by using the global network of functionally differentiated cortical regions. Hum Brain Mapp, 2013. © 2011 Wiley Periodicals, Inc.

[1]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[2]  Fumitaka Homae,et al.  Prosodic processing in the developing brain , 2007, Neuroscience Research.

[3]  Barbara Dodd,et al.  Lip reading in infants: Attention to speech presented in- and out-of-synchrony , 1979, Cognitive Psychology.

[4]  R. Andersen,et al.  Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. , 1996, Journal of neurophysiology.

[5]  J. Mehler,et al.  Sounds and silence: An optical topography study of language recognition at birth , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Shinsuke Shimojo,et al.  Development of multisensory spatial integration and perception in humans. , 2006, Developmental science.

[7]  Y. Benjamini,et al.  Adaptive linear step-up procedures that control the false discovery rate , 2006 .

[8]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[9]  C. Pierrot-Deseilligny,et al.  Eye movement control by the cerebral cortex , 2004, Current opinion in neurology.

[10]  Fumitaka Homae,et al.  Functional activation in diverse regions of the developing brain of human infants , 2008, NeuroImage.

[11]  M G Woldorff,et al.  Hemispheric asymmetries for different components of global/local attention occur in distinct temporo-parietal loci. , 2005, Cerebral cortex.

[12]  R. Adolphs The neurobiology of social cognition , 2001, Current Opinion in Neurobiology.

[13]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[14]  Archana K. Singh,et al.  Exploring the false discovery rate in multichannel NIRS , 2006, NeuroImage.

[15]  R. Lickliter,et al.  Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. , 2000, Developmental psychology.

[16]  Fumitaka Homae,et al.  Prefrontal cortical involvement in young infants' analysis of novelty. , 2009, Cerebral cortex.

[17]  S. Edelman,et al.  Cue-Invariant Activation in Object-Related Areas of the Human Occipital Lobe , 1998, Neuron.

[18]  Jonathan Berger,et al.  Neural Dynamics of Event Segmentation in Music: Converging Evidence for Dissociable Ventral and Dorsal Networks , 2007, Neuron.

[19]  G. Taga,et al.  Development of Global Cortical Networks in Early Infancy , 2010, The Journal of Neuroscience.

[20]  D. Lewkowicz,et al.  Sound induces perceptual reorganization of an ambiguous motion display in human infants , 2003 .

[21]  R. Tootell,et al.  Where is 'dorsal V4' in human visual cortex? Retinotopic, topographic and functional evidence. , 2001, Cerebral cortex.

[22]  Josh H. McDermott,et al.  Functional imaging of human visual recognition. , 1996, Brain research. Cognitive brain research.

[23]  Yumiko Otsuka,et al.  Neural activation to upright and inverted faces in infants measured by near infrared spectroscopy , 2007, NeuroImage.

[24]  Jean-Philippe Thiran,et al.  Multisensory interactions within human primary cortices revealed by BOLD dynamics. , 2007, Cerebral cortex.

[25]  E. Miller,et al.  THE PREFRONTAL CORTEX AND COGNITIVE CONTROL , 2000 .

[26]  Janette Atkinson,et al.  Regional Hemodynamic Responses to Visual Stimulation in Awake Infants , 1998, Pediatric Research.

[27]  Gentaro Taga,et al.  General to specific development of movement patterns and memory for contingency between actions and events in young infants. , 2006, Infant behavior & development.

[28]  R. Lickliter,et al.  The development of infant intersensory perception: advantages of a comparative convergent-operations approach. , 2000, Psychological bulletin.

[29]  K. Grill-Spector,et al.  The human visual cortex. , 2004, Annual review of neuroscience.

[30]  R. Turner,et al.  Detecting Latency Differences in Event-Related BOLD Responses: Application to Words versus Nonwords and Initial versus Repeated Face Presentations , 2002, NeuroImage.

[31]  N. Kanwisher,et al.  Visual attention: Insights from brain imaging , 2000, Nature Reviews Neuroscience.

[32]  I. Johnsrude,et al.  The problem of functional localization in the human brain , 2002, Nature Reviews Neuroscience.

[33]  E. Bullmore,et al.  Response amplification in sensory-specific cortices during crossmodal binding. , 1999, Neuroreport.

[34]  D. Munoz,et al.  Look away: the anti-saccade task and the voluntary control of eye movement , 2004, Nature Reviews Neuroscience.

[35]  Elizabeth S Spelke,et al.  Perception of Moving, Sounding Objects by Four-Month-Old Infants , 1983, Perception.

[36]  E. Watanabe,et al.  Spatial and temporal analysis of human motor activity using noninvasive NIR topography. , 1995, Medical physics.

[37]  N. Logothetis,et al.  Negative functional MRI response correlates with decreases in neuronal activity in monkey visual area V1 , 2006, Nature Neuroscience.

[38]  E. Miller,et al.  The prefontral cortex and cognitive control , 2000, Nature Reviews Neuroscience.

[39]  B. Hopkins,et al.  Developmental transformations of spontaneous movements in early infancy. , 1986, Early human development.

[40]  Michael S Beauchamp,et al.  See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex , 2005, Current Opinion in Neurobiology.

[41]  J. Rauschecker,et al.  Mechanisms and streams for processing of "what" and "where" in auditory cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[42]  T. Hendler,et al.  Convergence of visual and tactile shape processing in the human lateral occipital complex. , 2002, Cerebral cortex.

[43]  E. Spelke Infants' intermodal perception of events , 1976, Cognitive Psychology.

[44]  Klaus Scheffler,et al.  Differential patterns of multisensory interactions in core and belt areas of human auditory cortex , 2006, NeuroImage.

[45]  J. Colombo The development of visual attention in infancy. , 2001, Annual review of psychology.

[46]  R. Adolphs Cognitive neuroscience: Cognitive neuroscience of human social behaviour , 2003, Nature Reviews Neuroscience.

[47]  M. Cope,et al.  New non-invasive methods for assessing brain oxygenation and haemodynamics. , 1988, British medical bulletin.

[48]  A Villringer,et al.  Near-infrared spectroscopy: does it function in functional activation studies of the adult brain? , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[49]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[50]  David J. M. Kraemer,et al.  Musical imagery: Sound of silence activates auditory cortex , 2005, Nature.

[51]  Olivier Bertrand,et al.  Listening in Silence Activates Auditory Areas: A Functional Magnetic Resonance Imaging Study , 2006, The Journal of Neuroscience.

[52]  S. Zeki,et al.  Neural correlates of beauty. , 2004, Journal of neurophysiology.

[53]  Mark H. Johnson,et al.  Near infrared spectroscopy reveals neural activation during face perception in infants and adults , 2004, Journal of Pediatric Neurology.

[54]  Fumitaka Homae,et al.  Effects of source-detector distance of near infrared spectroscopy on the measurement of the cortical hemodynamic response in infants , 2007, NeuroImage.

[55]  Masako Okamoto,et al.  Three-dimensional probabilistic anatomical cranio-cerebral correlation via the international 10–20 system oriented for transcranial functional brain mapping , 2004, NeuroImage.

[56]  Ippeita Dan,et al.  Prefrontal activation associated with social attachment: facial-emotion recognition in mothers and infants. , 2009, Cerebral cortex.

[57]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[58]  J. Rieger,et al.  Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices , 2007, The Journal of Neuroscience.

[59]  Timothy S. Coalson,et al.  A Surface-Based Analysis of Hemispheric Asymmetries and Folding of Cerebral Cortex in Term-Born Human Infants , 2010, The Journal of Neuroscience.

[60]  Gentaro Taga,et al.  Selectivity and localization of cortical response to auditory and visual stimulation in awake infants aged 2 to 4 months , 2007, NeuroImage.

[61]  J. Gore,et al.  A Stimulus-Driven Approach to Object Identity and Location Processing in the Human Brain , 2000, Neuron.

[62]  M Castillo,et al.  Neonatal Localisation of a Sound in Visual Space , 1981, Perception.

[63]  Fumitaka Homae,et al.  The right hemisphere of sleeping infant perceives sentential prosody , 2006, Neuroscience Research.

[64]  J. Haxby,et al.  Parallel Visual Motion Processing Streams for Manipulable Objects and Human Movements , 2002, Neuron.

[65]  G. Taga,et al.  Brain imaging in awake infants by near-infrared optical topography , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[66]  A. Shmuel,et al.  Sustained Negative BOLD, Blood Flow and Oxygen Consumption Response and Its Coupling to the Positive Response in the Human Brain , 2002, Neuron.

[67]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[68]  K. D. Singh,et al.  Negative BOLD in the visual cortex: Evidence against blood stealing , 2004, Human brain mapping.

[69]  A. Amedi,et al.  Functional imaging of human crossmodal identification and object recognition , 2005, Experimental Brain Research.

[70]  D. Lewkowicz Heterogeneity and heterochrony in the development of intersensory perception. , 2002, Brain research. Cognitive brain research.

[71]  Kathleen S Rockland,et al.  Multisensory convergence in calcarine visual areas in macaque monkey. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[72]  E. Hartmann Perceptual Development: Visual, Auditory, and Speech Perception in Infancy , 1999 .

[73]  S. Hillyard,et al.  Delayed Striate Cortical Activation during Spatial Attention , 2002, Neuron.

[74]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[75]  Mark H. Johnson,et al.  Investigation of depth dependent changes in cerebral haemodynamics during face perception in infants , 2007, Physics in medicine and biology.

[76]  D. Heeger,et al.  Two Retinotopic Visual Areas in Human Lateral Occipital Cortex , 2006, The Journal of Neuroscience.

[77]  Scott Holland,et al.  Infant brain probability templates for MRI segmentation and normalization , 2008, NeuroImage.

[78]  Abraham Z. Snyder,et al.  The Feasibility of a Common Stereotactic Space for Children and Adults in fMRI Studies of Development , 2002, NeuroImage.

[79]  R. Malach,et al.  Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[80]  D. Kleinfeld,et al.  Suppressed Neuronal Activity and Concurrent Arteriolar Vasoconstriction May Explain Negative Blood Oxygenation Level-Dependent Signal , 2007, The Journal of Neuroscience.

[81]  J. Grafman,et al.  Human prefrontal cortex: processing and representational perspectives , 2003, Nature Reviews Neuroscience.

[82]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[83]  R. Zatorre,et al.  Where is 'where' in the human auditory cortex? , 2002, Nature Neuroscience.

[84]  Gentaro Taga,et al.  Flexibility in infant actions during arm- and leg-based learning in a mobile paradigm. , 2009, Infant behavior & development.

[85]  J. Mehler,et al.  The neonate brain detects speech structure , 2008, Proceedings of the National Academy of Sciences.

[86]  Masako Okamoto,et al.  Virtual spatial registration of stand-alone fNIRS data to MNI space , 2007, NeuroImage.

[87]  Z Kourtzi,et al.  Representation of Perceived Object Shape by the Human Lateral Occipital Complex , 2001, Science.

[88]  Brian L Allman,et al.  Multisensory processing in "unimodal" neurons: cross-modal subthreshold auditory effects in cat extrastriate visual cortex. , 2007, Journal of neurophysiology.

[89]  Fumitaka Homae,et al.  Anticipatory Cortical Activation Precedes Auditory Events in Sleeping Infants , 2008, PloS one.

[90]  Masako Okamoto,et al.  Automated cortical projection of head-surface locations for transcranial functional brain mapping , 2005, NeuroImage.

[91]  A. Villringer,et al.  Non-invasive optical spectroscopy and imaging of human brain function , 1997, Trends in Neurosciences.

[92]  David A. Boas,et al.  Assessing infants' cortical response to speech using near-infrared spectroscopy , 2007, NeuroImage.

[93]  Archana K. Singh,et al.  Virtual 10–20 measurement on MR images for inter-modal linking of transcranial and tomographic neuroimaging methods , 2005, NeuroImage.

[94]  Archana K. Singh,et al.  Spatial registration of multichannel multi-subject fNIRS data to MNI space without MRI , 2005, NeuroImage.

[95]  Y. Yen,et al.  Deactivation of Sensory-Specific Cortex by Cross-Modal Stimuli , 2002, Journal of Cognitive Neuroscience.

[96]  L. Bahrick Infants' perceptual differentiation of amodal and modality-specific audio-visual relations. , 1992, Journal of experimental child psychology.

[97]  G. Glover,et al.  Dissociated neural representations of intensity and valence in human olfaction , 2003, Nature Neuroscience.

[98]  M. Sigman,et al.  Functional organization of perisylvian activation during presentation of sentences in preverbal infants , 2006, Proceedings of the National Academy of Sciences.

[99]  J. Buhle,et al.  Typologies of attentional networks , 2006, Nature Reviews Neuroscience.

[100]  J. Gilmore,et al.  Infant Brain Atlases from Neonates to 1- and 2-Year-Olds , 2011, PloS one.

[101]  D. Lewkowicz Perception of auditory-visual temporal synchrony in human infants. , 1996, Journal of experimental psychology. Human perception and performance.

[102]  Christoph Kayser,et al.  Behavioral/systems/cognitive Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex , 2022 .

[103]  Koichi Mori,et al.  Neural Attunement Processes in Infants during the Acquisition of a Language-Specific Phonemic Contrast , 2007, The Journal of Neuroscience.

[104]  A. Slater Perceptual Development: Visual, Auditory, and Speech Perception in Infancy. Studies in Developmental Psychology. , 2000 .

[105]  D. Perrett,et al.  Dissociable neural responses to facial expressions of sadness and anger. , 1999, Brain : a journal of neurology.

[106]  T. Wilcox,et al.  Using near-infrared spectroscopy to assess neural activation during object processing in infants. , 2005, Journal of biomedical optics.

[107]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[108]  John G. Neuhoff,et al.  Spatiotemporal Pattern of Neural Processing in the Human Auditory Cortex , 2002, Science.

[109]  David J. Lewkowicz,et al.  Infants' response to temporally based intersensory equivalence: The effect of synchronous sounds on visual preferences for moving stimuli , 1992 .

[110]  Robert T. Knight,et al.  Temporal Characteristics of Audiovisual Information Processing , 2008, The Journal of Neuroscience.

[111]  Robert Lickliter,et al.  Infants’ perception of rhythm and tempo in unimodal and multimodal stimulation: A developmental test of the intersensory redundancy hypothesis , 2004, Cognitive, affective & behavioral neuroscience.

[112]  John J. Foxe,et al.  Crossmodal binding through neural coherence: implications for multisensory processing , 2008, Trends in Neurosciences.

[113]  E. Macaluso Multisensory Processing in Sensory-Specific Cortical Areas , 2006, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[114]  Alan C. Evans,et al.  Emotional responses to pleasant and unpleasant music correlate with activity in paralimbic brain regions , 1999, Nature Neuroscience.

[115]  Fumitaka Homae,et al.  General to specific development of functional activation in the cerebral cortexes of 2- to 3-month-old infants , 2010, NeuroImage.