Audiovisual temporal fusion in 6-month-old infants

The aim of this study was to investigate neural dynamics of audiovisual temporal fusion processes in 6-month-old infants using event-related brain potentials (ERPs). In a habituation-test paradigm, infants did not show any behavioral signs of discrimination of an audiovisual asynchrony of 200 ms, indicating perceptual fusion. In a subsequent EEG experiment, audiovisual synchronous stimuli and stimuli with a visual delay of 200 ms were presented in random order. In contrast to the behavioral data, brain activity differed significantly between the two conditions. Critically, N1 and P2 latency delays were not observed between synchronous and fused items, contrary to previously observed N1 and P2 latency delays between synchrony and perceived asynchrony. Hence, temporal interaction processes in the infant brain between the two sensory modalities varied as a function of perceptual fusion versus asynchrony perception. The visual recognition components Pb and Nc were modulated prior to sound onset, emphasizing the importance of anticipatory visual events for the prediction of auditory signals. Results suggest mechanisms by which young infants predictively adjust their ongoing neural activity to the temporal synchrony relations to be expected between vision and audition.

[1]  Franziska Kopp,et al.  Neural Dynamics of Audiovisual Synchrony and Asynchrony Perception in 6-Month-Old Infants , 2013, Front. Psychology.

[2]  Erich Schröger,et al.  The Human Brain Maintains Contradictory and Redundant Auditory Sensory Predictions , 2013, PloS one.

[3]  Katja Saupe,et al.  The N1-suppression effect for self-initiated sounds is independent of attention , 2013, BMC Neuroscience.

[4]  U. Lindenberger,et al.  Social cues at encoding affect memory in 4-month-old infants , 2012, Social neuroscience.

[5]  M. Wallace,et al.  Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. , 2012, Journal of experimental psychology. Human perception and performance.

[6]  V. Reid,et al.  Infant ERP amplitudes change over the course of an experimental session: Implications for cognitive processes and methodology , 2011, Brain and Development.

[7]  U. Lindenberger,et al.  Effects of joint attention on long-term memory in 9-month-old infants: an event-related potentials study. , 2011, Developmental science.

[8]  R. Flom,et al.  Neural signatures of face-voice synchrony in 5-month-old human infants. , 2011, Developmental psychobiology.

[9]  David B. Pisoni,et al.  Discrete neural substrates underlie complementary audiovisual speech integration processes , 2011, NeuroImage.

[10]  Albert R. Powers,et al.  Binding of sights and sounds: Age-related changes in multisensory temporal processing , 2011, Neuropsychologia.

[11]  D. Senkowski,et al.  The multifaceted interplay between attention and multisensory integration , 2010, Trends in Cognitive Sciences.

[12]  Jean Vroomen,et al.  Visual Anticipatory Information Modulates Multisensory Interactions of Artificial Audiovisual Stimuli , 2010, Journal of Cognitive Neuroscience.

[13]  Charles Spence,et al.  Perception of audiovisual speech synchrony for native and non-native language , 2010, Brain Research.

[14]  David B. Pisoni,et al.  Neural processing of asynchronous audiovisual speech perception , 2010, NeuroImage.

[15]  D. Lewkowicz,et al.  Intersensory Perception at Birth: Newborns Match Nonhuman Primate Faces and Voices. , 2010, Infancy : the official journal of the International Society on Infant Studies.

[16]  R. Flom,et al.  Visual stimulation enhances auditory processing in 3-month-old infants and adults. , 2009, Developmental psychobiology.

[17]  Olivier Bertrand,et al.  Electrophysiological (EEG, sEEG, MEG) evidence for multiple audiovisual interactions in the human auditory cortex , 2009, Hearing Research.

[18]  Albert R. Powers,et al.  Perceptual Training Narrows the Temporal Window of Multisensory Binding , 2009, The Journal of Neuroscience.

[19]  J. Colombo,et al.  Infant visual habituation , 2009, Neurobiology of Learning and Memory.

[20]  Michael Pilling.  Auditory event-related potentials (ERPs) in audiovisual speech perception. , 2009, Journal of speech, language, and hearing research : JSLHR.

[21]  Durk Talsma,et al.  Procedures and Strategies for Optimizing the Signal-to-Noise Ratio in Event-Related Potential Data , 2009 .

[22]  Daniel Senkowski,et al.  Intermodal attention affects the processing of the temporal alignment of audiovisual stimuli , 2009, Experimental Brain Research.

[23]  Jean-Francois Mangin,et al.  Hearing Faces: How the Infant Brain Matches the Face It Sees with the Speech It Hears , 2009, Journal of Cognitive Neuroscience.

[24]  Davide Rocchesso,et al.  Multisensory integration of drumming actions: musical expertise affects perceived audiovisual asynchrony , 2009, Experimental Brain Research.

[25]  D. Lewkowicz,et al.  The decline of cross-species intersensory perception in human infants: Underlying mechanisms and its developmental persistence , 2008, Brain Research.

[26]  P. Ackles.  Stimulus Novelty and Cognitive-Related Erp Components of the Infant Brain , 2008, Perceptual and motor skills.

[27]  D. Poeppel,et al.  Temporal window of integration in auditory-visual speech perception , 2007, Neuropsychologia.

[28]  Charles Spence,et al.  Temporal recalibration during asynchronous audiovisual speech perception , 2007, Experimental Brain Research.

[29]  Viktor K. Jirsa,et al.  Multisensory integration for timing engages different brain networks , 2007, NeuroImage.

[30]  P. Ackles,et al.  ATTENTION OR MEMORY? EFFECTS OF FAMILIARITY AND NOVELTY ON THE Nc COMPONENT OF EVENT-RELATED BRAIN POTENTIALS IN SIX-MONTH-OLD INFANTS , 2007, The International journal of neuroscience.

[31]  C. Spence,et al.  Audiovisual synchrony perception for music, speech, and object actions , 2006, Brain Research.

[32]  Jeffrey D Long,et al.  A longitudinal investigation of visual event-related potentials in the first year of life. , 2005, Developmental science.

[33]  Charles Spence,et al.  Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration. , 2005, Brain research. Cognitive brain research.

[34]  John E Richards,et al.  Familiarization, attention, and recognition memory in infancy: an event-related potential and cortical source localization study. , 2005, Developmental psychology.

[35]  Lee M. Miller,et al.  Behavioral/systems/cognitive Perceptual Fusion and Stimulus Coincidence in the Cross- Modal Integration of Speech , 2022 .

[36]  P. Jusczyk,et al.  Infants' use of synchronized visual information to separate streams of speech. , 2005, Child development.

[37]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[38]  A. Fort,et al.  Bimodal speech: early suppressive visual effects in human auditory cortex , 2004, The European journal of neuroscience.

[39]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[40]  E Macaluso,et al.  Spatial and temporal factors during processing of audiovisual speech: a PET study , 2004, NeuroImage.

[41]  C. Spence,et al.  Multisensory Integration: Maintaining the Perception of Synchrony , 2003, Current Biology.

[42]  John E Richards,et al.  Attention affects the recognition of briefly presented visual stimuli in infants: an ERP study. , 2003, Developmental science.

[43]  R Fendrich,et al.  The temporal cross-capture of audition and vision , 2001, Perception & psychophysics.

[44]  M. Hallett,et al.  Neural Correlates of Auditory–Visual Stimulus Onset Asynchrony Detection , 2001, The Journal of Neuroscience.

[45]  D. Lewkowicz,et al.  The development of intersensory temporal perception: an epigenetic systems/limitations view. , 2000, Psychological bulletin.

[46]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[47]  R. Karrer,et al.  Event-related potentials of 4-7-week-old infants in a visual recognition memory task. , 1995, Electroencephalography and clinical neurophysiology.

[48]  David J. Lewkowicz,et al.  Infants' response to temporally based intersensory equivalence: The effect of synchronous sounds on visual preferences for moving stimuli , 1992 .

[49]  F. Perrin,et al.  Spherical splines for scalp potential and current density mapping. , 1989, Electroencephalography and clinical neurophysiology.

[50]  L. Bahrick.  Infants' intermodal perception of two levels of temporal structure in natural events☆ , 1987 .

[51]  David J. Lewkowicz,et al.  Developmental changes in infants' bisensory response to synchronous durations☆ , 1986 .

[52]  Lorraine E. Bahrick,et al.  Infants' perception of substance and temporal synchrony in multimodal events* , 1983 .

[53]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[54]  E. Spelke.  Perceiving Bimodally Specified Events in Infancy , 1979 .

[55]  Barbara Dodd,et al.  Lip reading in infants: Attention to speech presented in- and out-of-synchrony , 1979, Cognitive Psychology.

[56]  Murray Mm,et al.  Multisensory Integration Develops Late in Humans -- The Neural Bases of Multisensory Processes , 2012 .

[57]  R. Aslin.  What's in a look? , 2007, Developmental science.

[58]  P. Bauer,et al.  Infants' ERP Responses to Novel and Familiar Stimuli Change Over Time: Implications for Novelty Detection and Memory , 2006 .

[59]  A. King,et al.  Multisensory Integration: Strategies for Synchronization , 2005, Current Biology.

[60]  D. Lewkowicz.  Perception of auditory-visual temporal synchrony in human infants. , 1996, Journal of experimental psychology. Human perception and performance.

[61]  R. Karrer,et al.  Differential effects of experience on the ERP and behavior of 6-month-old infants: trends during repeated stimulus presentations , 1994 .