Neural development of networks for audiovisual speech comprehension

Everyday conversation is both an auditory and a visual phenomenon. While visual speech information enhances comprehension for the listener, evidence suggests that the ability to benefit from this information improves with development. A number of brain regions have been implicated in audiovisual speech comprehension, but the extent to which the neurobiological substrate in the child compares to the adult is unknown. In particular, developmental differences in the network for audiovisual speech comprehension could manifest through the incorporation of additional brain regions, or through different patterns of effective connectivity. In the present study we used functional magnetic resonance imaging and structural equation modeling (SEM) to characterize the developmental changes in network interactions for audiovisual speech comprehension. The brain response was recorded while children 8- to 11-years-old and adults passively listened to stories under audiovisual (AV) and auditory-only (A) conditions. Results showed that in children and adults, AV comprehension activated the same fronto-temporo-parietal network of regions known for their contribution to speech production and perception. However, the SEM network analysis revealed age-related differences in the functional interactions among these regions. In particular, the influence of the posterior inferior frontal gyrus/ventral premotor cortex on supramarginal gyrus differed across age groups during AV, but not A speech. This functional pathway might be important for relating motor and sensory information used by the listener to identify speech sounds. Further, its development might reflect changes in the mechanisms that relate visual speech information to articulatory speech representations through experience producing and perceiving speech.

[1]  A. Meltzoff,et al.  The Intermodal Representation of Speech in Infants , 1984 .

[2]  Michael Wilde,et al.  Improving the analysis, storage and sharing of neuroimaging data using relational databases and distributed computing , 2008, NeuroImage.

[3]  Jeffery A. Jones,et al.  Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory–auditory/orosensory internal models , 2004, NeuroImage.

[4]  Y. Sugita,et al.  Auditory-visual speech perception examined by fMRI and PET , 2003, Neuroscience Research.

[5]  G. Dawson,et al.  The role of early experience in shaping behavioral and brain development and its implications for social policy , 2000, Development and Psychopathology.

[6]  Q Summerfield,et al.  Use of Visual Information for Phonetic Perception , 1979, Phonetica.

[7]  G. Westermann,et al.  A new model of sensorimotor coupling in the development of speech , 2004, Brain and Language.

[8]  Scott K Holland,et al.  Practical Aspects of Conducting Large-Scale Functional Magnetic Resonance Imaging Studies in Children , 2002, Journal of child neurology.

[9]  Karl Magnus Petersson,et al.  Language Processing Modulated by Literacy: A Network Analysis of Verbal Repetition in Literate and Illiterate Subjects , 2000, Journal of Cognitive Neuroscience.

[10]  F. Duffy,et al.  Early experience alters brain function and structure. , 2004, Pediatrics.

[11]  David B Pisoni,et al.  Development of Audiovisual Comprehension Skills in Prelingually Deaf Children With Cochlear Implants , 2005, Ear and hearing.

[12]  Vincent Schmithorst,et al.  Cognitive modules utilized for narrative comprehension in children: a functional magnetic resonance imaging study , 2006, NeuroImage.

[13]  Brian MacWhinney,et al.  The emergence of language. , 1999 .

[14]  Jeffery A. Jones,et al.  Brain activity during audiovisual speech perception: An fMRI study of the McGurk effect , 2003, Neuroreport.

[15]  F. Guenther Cortical interactions underlying the production of speech sounds. , 2006, Journal of communication disorders.

[16]  P. Barrett Structural equation modelling : Adjudging model fit , 2007 .

[17]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[18]  Lawrence Brancazio,et al.  Use of visual information in speech perception: Evidence for a visual rate effect both with and without a McGurk effect , 2005, Perception & psychophysics.

[19]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Marco Iacoboni,et al.  Neural responses to non-native phonemes varying in producibility: Evidence for the sensorimotor nature of speech perception , 2006, NeuroImage.

[21]  Kayoko Okada,et al.  Two cortical mechanisms support the integration of visual and auditory speech: A hypothesis and preliminary data , 2009, Neuroscience Letters.

[22]  S. Small,et al.  Fine modulation in network activation during motor execution and motor imagery. , 2004, Cerebral cortex.

[23]  Suzanne E. Welcome,et al.  Longitudinal Mapping of Cortical Thickness and Brain Growth in Normal Children , 2022 .

[24]  E. Darcy Burgund,et al.  Comparison of functional activation foci in children and adults using a common stereotactic space , 2003, NeuroImage.

[25]  Jeremy I. Skipper,et al.  Gestures Orchestrate Brain Networks for Language Understanding , 2009, Current Biology.

[26]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[27]  S. Petersen,et al.  Development of distinct control networks through segregation and integration , 2007, Proceedings of the National Academy of Sciences.

[28]  O. Bertrand,et al.  Visual Activation and Audiovisual Interactions in the Auditory Cortex during Speech Perception: Intracranial Recordings in Humans , 2008, The Journal of Neuroscience.

[29]  Louis-Jean Boë,et al.  Role of vocal tract morphology in speech development: perceptual targets and sensorimotor maps for synthesized French vowels from birth to adulthood. , 2004, Journal of speech, language, and hearing research : JSLHR.

[30]  L. Rosenblum,et al.  The McGurk effect in infants , 1997 .

[31]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[32]  Jeremy I. Skipper,et al.  Action to Language via the Mirror Neuron System: Lending a helping hand to hearing: another motor theory of speech perception , 2006 .

[33]  Barbara Dodd,et al.  Lip reading in infants: Attention to speech presented in- and out-of-synchrony , 1979, Cognitive Psychology.

[34]  Vittorio Gallese,et al.  Mirror Neurons and the Evolution of Brain and Language , 2002 .

[35]  B. Argall,et al.  Simplified intersubject averaging on the cortical surface using SUMA , 2006, Human brain mapping.

[36]  Leslie G. Ungerleider,et al.  Network analysis of cortical visual pathways mapped with PET , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  M. Vihman The role of mirror neurons in the ontogeny of speech , 2008 .

[38]  J. Schwartz,et al.  Seeing to hear better: evidence for early audio-visual interactions in speech identification , 2004, Cognition.

[39]  H. Duffau,et al.  The articulatory loop: study of the subcortical connectivity by electrostimulation , 2003, Neuroreport.

[40]  G. Bi,et al.  Distributed synaptic modification in neural networks induced by patterned stimulation , 1999, Nature.

[41]  Louis-Jean Boë,et al.  Articulatory-acoustic relationships during vocal tract growth for French vowels: Analysis of real data and simulations with an articulatory model , 2007, J. Phonetics.

[42]  Friedemann Pulvermüller,et al.  Motor cortex maps articulatory features of speech sounds , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[43]  June Yoshii,et al.  Age-related changes in prefrontal white matter volume across adolescence , 2006, Neuroreport.

[44]  Marco Iacoboni,et al.  The Essential Role of Premotor Cortex in Speech Perception , 2007, Current Biology.

[45]  Todd B. Parrish,et al.  Impact of signal‐to‐noise on functional MRI , 2000 .

[46]  Whitney M. Weikum,et al.  Visual language discrimination , 2008 .

[47]  J. Werker,et al.  An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks. , 1997, Journal of experimental child psychology.

[48]  N. Makris,et al.  A methodology for analyzing curvature in the developing brain from preterm to adult , 2008, Int. J. Imaging Syst. Technol..

[49]  M. Arbib Action to language via the mirror neuron system , 2006 .

[50]  J. Gore,et al.  A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. , 2002, Brain research. Cognitive brain research.

[51]  D. Burnham,et al.  Impact of language on development of auditory-visual speech perception. , 2008, Developmental science.

[52]  Abraham Z. Snyder,et al.  The Feasibility of a Common Stereotactic Space for Children and Adults in fMRI Studies of Development , 2002, NeuroImage.

[53]  L. Bernstein,et al.  Quantified acoustic–optical speech signal incongruity identifies cortical sites of audiovisual speech processing , 2008, Brain Research.

[54]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[55]  Virginie van Wassenhove,et al.  Auditory-visual fusion in speech perception in children with cochlear implants. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Frank H. Guenther,et al.  An fMRI investigation of syllable sequence production , 2006, NeuroImage.

[57]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[58]  Jeremy I. Skipper,et al.  Co‐speech gestures influence neural activity in brain regions associated with processing semantic information , 2009, Human brain mapping.

[59]  Nikos Makris,et al.  Automatically parcellating the human cerebral cortex. , 2004, Cerebral cortex.

[60]  Y. Tohkura,et al.  Inter-language differences in the influence of visual cues in speech perception. , 1993 .

[61]  Anthony D Wagner,et al.  Assembling and encoding word representations: fMRI subsequent memory effects implicate a role for phonological control , 2003, Neuropsychologia.

[62]  Alan C. Evans,et al.  Brain development during childhood and adolescence: a longitudinal MRI study , 1999, Nature Neuroscience.

[63]  P S Goldman-Rakic,et al.  Development of cortical circuitry and cognitive function. , 1987, Child development.

[64]  Mikko Sams,et al.  Processing of audiovisual speech in Broca's area , 2005, NeuroImage.

[65]  Thomas E. Nichols,et al.  Validating cluster size inference: random field and permutation methods , 2003, NeuroImage.

[66]  Raymond D. Kent,et al.  An auditory-feedback-based neural network model of speech production that is robust to developmental changes in the size and shape of the articulatory system. , 2000, Journal of speech, language, and hearing research : JSLHR.

[67]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[68]  Ruth Campbell,et al.  The processing of audio-visual speech: empirical and neural bases , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[69]  G Ojemann,et al.  Localization of language in frontal cortex. , 1992, Advances in neurology.

[70]  Satrajit S. Ghosh,et al.  Neural modeling and imaging of the cortical interactions underlying syllable production , 2006, Brain and Language.

[71]  F. Gonzalez-Lima,et al.  Structural equation modeling and its application to network analysis in functional brain imaging , 1994 .

[72]  D. Pandya,et al.  Fiber Pathways of the Brain , 2006 .

[73]  Denis Burnham,et al.  Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect. , 2004, Developmental psychobiology.

[74]  S. Petersen,et al.  Control networks in paediatric Tourette syndrome show immature and anomalous patterns of functional connectivity , 2008, Brain : a journal of neurology.

[75]  L. Westlye,et al.  Brain maturation in adolescence and young adulthood: regional age-related changes in cortical thickness and white matter volume and microstructure. , 2010, Cerebral cortex.

[76]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[77]  Vincent Schmithorst,et al.  Age-related connectivity changes in fMRI data from children listening to stories , 2007, NeuroImage.

[78]  R. Aslin,et al.  Visual speech contributes to phonetic learning in 6-month-old infants , 2008, Cognition.

[79]  S. Petersen,et al.  The maturing architecture of the brain's default network , 2008, Proceedings of the National Academy of Sciences.

[80]  P. Huttenlocher Synaptic density in human frontal cortex - developmental changes and effects of aging. , 1979, Brain research.

[81]  C. Price,et al.  The role of the posterior superior temporal sulcus in audiovisual processing. , 2008, Cerebral cortex.

[82]  Jeffery A. Jones,et al.  Neural processes underlying perceptual enhancement by visual speech gestures , 2003, Neuroreport.

[83]  J. Vroomen,et al.  Audiovisual speech recalibration in children* , 2008, Journal of Child Language.

[84]  M. Sams,et al.  Attention to visual speech gestures enhances hemodynamic activity in the left planum temporale , 2006, Human brain mapping.

[85]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[86]  David C. Plaut,et al.  The emergence of phonology from the interplay of speech comprehension and production ; A distributed connectionist approach , 1998 .

[87]  Istvan Molnar-Szakacs,et al.  Beyond superior temporal cortex: intersubject correlations in narrative speech comprehension. , 2008, Cerebral cortex.

[88]  Tom Johnstone,et al.  Motion correction and the use of motion covariates in multiple‐subject fMRI analysis , 2006, Human brain mapping.

[89]  H. McGurk,et al.  La langue et les Lèvres: cross-language influences on bimodal speech perception. , 1992, Canadian journal of psychology.

[90]  Steven L. Small,et al.  Abstract Coding of Audiovisual Speech: Beyond Sensory Representation , 2007, Neuron.

[91]  E. Cabanis,et al.  The Human Brain: Surface, Three-Dimensional Sectional Anatomy and Mri , 1991 .

[92]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[93]  P. McGuire,et al.  Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). , 2001, Brain research. Cognitive brain research.

[94]  Jeffrey M. Zacks,et al.  Pictures of a thousand words: Investigating the neural mechanisms of reading with extremely rapid event-related fMRI , 2008, NeuroImage.

[95]  J. Werker,et al.  Two-month-old infants match phonetic information in lips and voice , 2003 .

[96]  J. Cohen,et al.  Spiral K‐space MR imaging of cortical activation , 1995, Journal of magnetic resonance imaging : JMRI.

[97]  A A Montgomery,et al.  Auditory and visual contributions to the perception of consonants. , 1974, Journal of speech and hearing research.

[98]  Maurizio Corbetta,et al.  The human brain is intrinsically organized into dynamic, anticorrelated functional networks. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[99]  D B Pisoni,et al.  Use of Audiovisual Information in Speech Perception by Prelingually Deaf Children with Cochlear Implants: A First Report , 2001, Ear and hearing.

[100]  Jeremy I. Skipper,et al.  Speech-associated gestures, Broca’s area, and the human mirror system , 2007, Brain and Language.

[101]  P. Huttenlocher,et al.  Regional differences in synaptogenesis in human cerebral cortex , 1997, The Journal of comparative neurology.

[102]  Jeremy I. Skipper,et al.  Seeing Voices : How Cortical Areas Supporting Speech Production Mediate Audiovisual Speech Perception , 2007 .

[103]  S. M. Daselaar,et al.  When less means more: deactivations during encoding that predict subsequent memory , 2004, NeuroImage.

[104]  A. Toga,et al.  In vivo evidence for post-adolescent brain maturation in frontal and striatal regions , 1999, Nature Neuroscience.

[105]  Lee M. Miller,et al.  Behavioral/systems/cognitive Perceptual Fusion and Stimulus Coincidence in the Cross- Modal Integration of Speech , 2022 .

[106]  Mark S. Cohen,et al.  Parametric Analysis of fMRI Data Using Linear Systems Methods , 1997, NeuroImage.

[107]  James L. Arbuckle AMOS: Analysis of Moment Structures , 1989 .

[108]  R. Campbell,et al.  Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech , 2003, Journal of Cognitive Neuroscience.

[109]  Vincent L. Gracco,et al.  A mediating role of the premotor cortex in phoneme segmentation , 2009, Brain and Language.

[110]  A. McIntosh,et al.  Structural modeling of functional visual pathways mapped with 2-deoxyglucose: effects of patterned light and footshock , 1992, Brain Research.

[111]  T. Paus Mapping brain maturation and cognitive development during adolescence , 2005, Trends in Cognitive Sciences.

[112]  Thomas F. Nugent,et al.  Dynamic mapping of human cortical development during childhood through early adulthood. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[113]  J. Jaccard Interaction Effects in Factorial Analysis of Variance , 1997 .

[114]  Natalie M. Myres,et al.  Distinctive Paleo-Indian Migration Routes from Beringia Marked by Two Rare mtDNA Haplogroups , 2009, Current Biology.

[115]  M. Corbetta,et al.  Common Blood Flow Changes across Visual Tasks: II. Decreases in Cerebral Cortex , 1997, Journal of Cognitive Neuroscience.

[116]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[117]  P. Yakovlev,et al.  The myelogenetic cycles of regional maturation of the brain , 1967 .

[118]  C. Shatz,et al.  Synaptic Activity and the Construction of Cortical Circuits , 1996, Science.

[119]  Gregory McCarthy,et al.  Polysensory interactions along lateral temporal regions evoked by audiovisual speech. , 2003, Cerebral cortex.

[120]  Uri Hasson,et al.  Brain networks subserving the extraction of sentence information and its encoding to memory. , 2007, Cerebral cortex.

[121]  D W Massaro,et al.  Children's perception of visual and auditory speech. , 1984, Child development.

[122]  A. Meltzoff,et al.  The bimodal perception of speech in infancy. , 1982, Science.

[123]  R. C. Oldfield THE ASSESSMENT AND ANALYSIS OF HANDEDNESS , 1971 .

[124]  Jason A. Tourville,et al.  Neural mechanisms underlying auditory feedback control of speech , 2008, NeuroImage.

[125]  A. Minkowski,et al.  Regional Development of the Brain in Early Life , 1968 .

[126]  J. Werker,et al.  Is the integration of heard and seen speech mandatory for infants? , 2004, Developmental psychobiology.

[127]  Whitney M. Weikum,et al.  Visual Language Discrimination in Infancy , 2007, Science.

[128]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[129]  D. Burnham,et al.  Perception of the auditory‐visual illusion in speech perception by children with phonological disorders , 2008, Clinical linguistics & phonetics.

[130]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[131]  Raymond D. Kent,et al.  Development of vocal tract length during early childhood: a magnetic resonance imaging study. , 2005, The Journal of the Acoustical Society of America.

[132]  L A Thompson,et al.  Developmental changes in visual and auditory contributions to speech perception. , 1986, Journal of experimental child psychology.

[133]  Linda Polka,et al.  A developmental study of audiovisual speech perception using the McGurk paradigm , 1994 .

[134]  H. Nusbaum,et al.  On the neurobiological investigation of language understanding in context , 2004, Brain and Language.

[135]  Manabu Honda,et al.  Cross-modal binding and activated attentional networks during audio-visual speech integration: a functional MRI study. , 2005, Cerebral cortex.

[136]  D. Magnusson The lifespan development of individuals : behavioral, neurobiological, and psychosocial perspectives : a synthesis , 1997 .

[137]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[138]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[139]  Keiichi Tajima,et al.  Neural processes involved with perception of non-native durational contrasts , 2006, Neuroreport.

[140]  Douglas Brungart,et al.  Informational masking of speech in children: auditory-visual integration. , 2006, The Journal of the Acoustical Society of America.

[141]  Steven L. Small,et al.  Listening to talking faces: motor cortical activation during speech perception , 2005, NeuroImage.