Multimodal lexical processing in auditory cortex is literacy skill dependent.

Literacy is a uniquely human cross-modal cognitive process wherein visual orthographic representations become associated with auditory phonological representations through experience. Developmental studies provide insight into how experience-dependent changes in brain organization influence phonological processing as a function of literacy. Previous investigations show a synchrony-dependent influence of letter presentation on individual phoneme processing in superior temporal sulcus; others demonstrate recruitment of primary and associative auditory cortex during cross-modal processing. We sought to determine whether brain regions supporting phonological processing of larger lexical units (monosyllabic words) over larger time windows is sensitive to cross-modal information, and whether such effects are literacy dependent. Twenty-two children (age 8-14 years) made rhyming judgments for sequentially presented word and pseudoword pairs presented either unimodally (auditory- or visual-only) or cross-modally (audiovisual). Regression analyses examined the relationship between literacy and congruency effects (overlapping orthography and phonology vs. overlapping phonology-only). We extend previous findings by showing that higher literacy is correlated with greater congruency effects in auditory cortex (i.e., planum temporale) only for cross-modal processing. These skill effects were specific to known words and occurred over a large time window, suggesting that multimodal integration in posterior auditory cortex is critical for fluent reading.

[1]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.

[2]  Nancy Mather,et al.  Woodcock-Johnson III Tests of Cognitive Abilities. , 2009 .

[3]  Josef P. Rauschecker,et al.  An expanded role for the dorsal auditory pathway in sensorimotor control and integration , 2011, Hearing Research.

[4]  Audrey R. Nath,et al.  fMRI-Guided Transcranial Magnetic Stimulation Reveals That the Superior Temporal Sulcus Is a Cortical Locus of the McGurk Effect , 2010, The Journal of Neuroscience.

[5]  E. Darcy Burgund,et al.  Comparison of functional activation foci in children and adults using a common stereotactic space , 2003, NeuroImage.

[6]  S. Dehaene,et al.  Functional Neuroimaging of Speech Perception in Infants , 2002, Science.

[7]  P. Kuhl.  Early language acquisition: cracking the speech code , 2004, Nature Reviews Neuroscience.

[8]  Robert T. Knight,et al.  Superior Temporal SulcusIt's My Area: Or Is It? , 2008, Journal of Cognitive Neuroscience.

[9]  Gregory Hickok,et al.  The functional neuroanatomy of language. , 2009, Physics of life reviews.

[10]  Bradley R. Buchsbaum,et al.  The Search for the Phonological Store: From Loop to Convolution , 2008, Journal of Cognitive Neuroscience.

[11]  Lee M. Miller,et al.  Behavioral/systems/cognitive Perceptual Fusion and Stimulus Coincidence in the Cross- Modal Integration of Speech , 2022 .

[12]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[13]  Rainer Goebel,et al.  Interaction of speech and script in human auditory cortex: Insights from neuro-imaging and effective connectivity , 2009, Hearing Research.

[14]  James R. Booth,et al.  Functional Anatomy of Intra- and Cross-Modal Lexical Tasks , 2002, NeuroImage.

[15]  Dries J. W. Froyen,et al.  Cross-modal enhancement of the MMN to speech-sounds indicates early and automatic integration of letters and speech-sounds , 2008, Neuroscience Letters.

[16]  J. Ziegler,et al.  Reading acquisition, developmental dyslexia, and skilled reading across languages: a psycholinguistic grain size theory. , 2005, Psychological bulletin.

[17]  John O. Willis,et al.  Wechsler Abbreviated Scale of Intelligence , 2014 .

[18]  E. Bullmore,et al.  Activation of auditory cortex during silent lipreading. , 1997, Science.

[19]  Jason A. Tourville,et al.  The integration of large-scale neural network modeling and functional brain imaging in speech motor control , 2010, NeuroImage.

[20]  M. Coltheart,et al.  Surface dyslexia. , 1983, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[21]  Ryan A. Stevenson,et al.  Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition , 2009, NeuroImage.

[22]  Michael S Beauchamp,et al.  Neural Correlates of Interindividual Differences in Children's Audiovisual Speech Perception , 2011, The Journal of Neuroscience.

[23]  Christoph Kayser,et al.  Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex , 2007, The Journal of Neuroscience.

[24]  Kayoko Okada,et al.  Area Spt in the Human Planum Temporale Supports Sensory-motor Integration for Speech Processing Establishing the Existence of Distinct Sen- Sory versus Motor Activation Patterns Would Establish That , 2022 .

[25]  Johannes C. Ziegler,et al.  Nonword reading across orthographies: How flexible is the choice of reading units? , 2003, Applied Psycholinguistics.

[26]  Rainer Goebel,et al.  The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. , 2006, Cerebral cortex.

[27]  N. Logothetis,et al.  Visual modulation of neurons in auditory cortex. , 2008, Cerebral cortex.

[28]  R. Goebel,et al.  Reduced Neural Integration of Letters and Speech Sounds Links Phonological and Reading Deficits in Adult Dyslexia , 2009, Current Biology.

[29]  J. Kaiser,et al.  Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas , 2007, The Journal of Neuroscience.

[30]  C. McNorgan,et al.  Prediction of Reading Skill Several Years Later Depends on Age and Brain Region: Implications for Developmental Models of Reading , 2011, The Journal of Neuroscience.

[31]  James R. Booth,et al.  Development of Brain Mechanisms for Processing Orthographic and Phonologic Representations , 2004, Journal of Cognitive Neuroscience.

[32]  D. Baxter.  Surface Dyslexia: Neuropsychological and Cognitive Studies of Phonological Reading , 1987 .

[33]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[34]  J. Fletcher,et al.  Prevalence of reading disability in boys and girls. Results of the Connecticut Longitudinal Study. , 1990, JAMA.

[35]  J L Lancaster,et al.  Automated Talairach Atlas labels for functional brain mapping , 2000, Human brain mapping.

[36]  Mark S. Seidenberg,et al.  Phonology, reading acquisition, and dyslexia: insights from connectionist models. , 1999, Psychological review.

[37]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[38]  Kevin R. Smith,et al.  Word length modulates neural activity in auditory cortex during covert object naming , 2003, Neuroreport.

[39]  Rainer Goebel,et al.  Top–down task effects overrule automatic multisensory responses to letter–sound pairs in auditory association cortex , 2006, NeuroImage.

[40]  Michael S. Beauchamp,et al.  A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion , 2012, NeuroImage.

[41]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[42]  D. Poeppel,et al.  The Human Auditory Cortex , 2012, Springer Handbook of Auditory Research.

[43]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[44]  Dries J. W. Froyen,et al.  The Long Road to Automation: Neurocognitive Development of Letter–Speech Sound Processing , 2009, Journal of Cognitive Neuroscience.

[45]  Tali Bitan,et al.  The interaction between orthographic and phonological information in children: An fMRI study , 2007, Human brain mapping.

[46]  A. Baddeley.  Working memory: theories, models, and controversies. , 2012, Annual review of psychology.

[47]  Bradley R. Buchsbaum,et al.  Reading, hearing, and the planum temporale , 2005, NeuroImage.

[48]  Bruce D. McCandliss,et al.  Development of neural systems for reading. , 2007, Annual review of neuroscience.

[49]  Ron Dumont,et al.  Woodcock-Johnson III Tests Of Cognitive Abilities , 2014 .

[50]  J. Theeuwes,et al.  Attention and the multiple stages of multisensory integration: A review of audiovisual studies. , 2010, Acta psychologica.

[51]  Abraham Z. Snyder,et al.  The Feasibility of a Common Stereotactic Space for Children and Adults in fMRI Studies of Development , 2002, NeuroImage.

[52]  Gregory Hickok,et al.  Redefining the Functional Organization of the Planum Temporale Region: Space, Objects, and Sensory–Motor Integration , 2012 .

[53]  John J. Foxe,et al.  The development of multisensory speech perception continues into the late childhood years , 2011, The European journal of neuroscience.

[54]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[55]  G. Calvert.  Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[56]  G. Hickok.  The cortical organization of speech processing: feedback control and predictive coding the context of a dual-stream model. , 2012, Journal of communication disorders.