Cortical tracking of speech in noise accounts for reading strategies in children

Humans’ propensity to acquire literacy relates to several factors, among which, the ability to understand speech in noise (SiN). Still, the nature of the relation between reading and SiN perception abilities remains poorly understood. Here, we dissect the interplay between (i) reading abilities, (ii) classical behavioral predictors of reading (phonological awareness, phonological memory and lexical access), and (iii) electrophysiological markers of SiN perception in 99 elementary school children (26 with dyslexia). We demonstrate that cortical representation of phrasal content of SiN relates to the development of the lexical (but not sublexical) reading strategy. In contrast, classical behavioral predictors of reading abilities and the ability to benefit from visual speech to represent the syllabic content of SiN account for global reading performance (i.e., speed and accuracy of lexical and sublexical reading). Finally, we found that individuals with dyslexia properly integrate visual speech information to optimize processing of syntactic information, but not to sustain acoustic/phonemic processing. These results clarify the nature of the relation between SiN perception and reading abilities in typical and dyslexic child readers, and identified novel electrophysiological markers of emergent literacy.

[1]  Mathieu Bourguignon,et al.  Comparing the potential of MEG and EEG to uncover brain tracking of speech temporal envelope , 2019, NeuroImage.

[2]  Riitta Hari,et al.  Cortical Tracking of Speech-in-Noise Develops from Childhood to Adulthood , 2019, The Journal of Neuroscience.

[3]  Lars Meyer,et al.  Synchronization of Electrophysiological Responses with Speech Benefits Syntactic Information Processing , 2018, Journal of Cognitive Neuroscience.

[4]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[5]  N. Mesgarani,et al.  Selective cortical representation of attended speaker in multi-talker speech perception , 2012, Nature.

[6]  M. Wolf,et al.  Rapid automatized naming (RAN) and reading fluency: implications for understanding and treatment of reading disabilities. , 2012, Annual review of psychology.

[7]  D. Poeppel,et al.  Cortical Tracking of Hierarchical Linguistic Structures in Connected Speech , 2015, Nature Neuroscience.

[8]  T. Lachmann,et al.  Does noise affect learning? A short review on noise effects on cognitive performance in children , 2013, Front. Psychol..

[9]  Charles A. Perfetti,et al.  Phonemic knowledge and learning to read are reciprocal: A longitudinal study of first grade children. , 1987 .

[10]  C. Colin,et al.  Atypical audio-visual speech perception and McGurk effects in children with specific language impairment , 2014, Front. Psychol..

[11]  Robin A. A. Ince,et al.  Healthy ageing delays the neural processing of face features relevant for behaviour by 40 ms , 2019 .

[12]  Robin A. A. Ince Measuring multivariate redundant information with pointwise common change in surprisal , 2016, Entropy.

[13]  Sarah M. N. Woolley,et al.  A Behavioral Framework to Guide Research on Central Auditory Development and Plasticity , 2011, Neuron.

[14]  C. Currie,et al.  Indicators of socioeconomic status for adolescents: the WHO Health Behaviour in School-aged Children Survey. , 1997, Health education research.

[15]  Bruce D. McCandliss,et al.  The visual word form area: expertise for reading in the fusiform gyrus , 2003, Trends in Cognitive Sciences.

[16]  Ruth Campbell,et al.  The processing of audio-visual speech: empirical and neural bases , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[17]  James L. McClelland,et al.  The parallel distributed processing approach to semantic cognition , 2003, Nature Reviews Neuroscience.

[18]  Cathy J. Price,et al.  A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading , 2012, NeuroImage.

[19]  Fanny Meunier,et al.  Speech-in-noise perception deficit in adults with dyslexia: Effects of background type and listening configuration , 2012, Neuropsychologia.

[20]  Edmund C. Lalor,et al.  Visual Cortical Entrainment to Motion and Categorical Speech Features during Silent Lipreading , 2017, Front. Hum. Neurosci..

[21]  Bettina Sorger,et al.  Neural Entrainment to Speech Modulates Speech Intelligibility , 2017, Current Biology.

[22]  Alexander Bertrand,et al.  Auditory-Inspired Speech Envelope Extraction Methods for Improved EEG-Based Auditory Attention Detection in a Cocktail Party Scenario , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  R. Freyman,et al.  The role of visual speech cues in reducing energetic and informational masking. , 2005, The Journal of the Acoustical Society of America.

[24]  Josh H. McDermott The cocktail party problem , 2009, Current Biology.

[25]  E. Liebenthal,et al.  Neural pathways for visual speech perception , 2014, Front. Neurosci..

[26]  P. Schyns,et al.  Speech Rhythms and Multiplexed Oscillatory Sensory Coding in the Human Brain , 2013, PLoS biology.

[27]  J. Sideris,et al.  Longitudinal predictors of reading and math trajectories through middle school for African American versus Caucasian students across two samples. , 2010, Developmental psychology.

[28]  S. Mattys,et al.  Effects of cognitive load on speech recognition , 2011 .

[29]  Donald Shankweiler The Speech Code and Learning to Read. , 1979 .

[30]  E. Conlon,et al.  Visual and auditory temporal processing and early reading development. , 2004, Dyslexia.

[31]  Robin A. A. Ince,et al.  Representational interactions during audiovisual speech entrainment: Redundancy in left posterior superior temporal gyrus and synergy in left motor cortex , 2018, PLoS biology.

[32]  Stefano Panzeri,et al.  Contributions of local speech encoding and functional connectivity to audio-visual speech perception , 2017, eLife.

[33]  V. Muter,et al.  Concurrent and Longitudinal Predictors of Reading: The Role of Metalinguistic and Short‐Term Memory Skills , 1998 .

[34]  Gregory B. Cogan,et al.  Visual Input Enhances Selective Speech Envelope Tracking in Auditory Cortex at a “Cocktail Party” , 2013, The Journal of Neuroscience.

[35]  V. Jousmäki,et al.  The pace of prosodic phrasing couples the listener's cortex to the reader's voice , 2013, Human brain mapping.

[36]  Stephen R. Burgess,et al.  Contributions of Phonological Awareness and Rapid Automatic Naming Ability to the Growth of Word-Reading Skills in Second- to Fifth-Grade Children. , 1997 .

[37]  Karl Magnus Petersson,et al.  Rapid Automatized Naming and Reading Performance: A Meta-Analysis. , 2014 .

[38]  U. Goswami Sensory theories of developmental dyslexia: three challenges for research , 2014, Nature Reviews Neuroscience.

[39]  T. Münte,et al.  Audiovisual perception of natural speech is impaired in adult dyslexics: An ERP study , 2015, Neuroscience.

[40]  Guillaume A. Rousselet,et al.  A statistical framework for neuroimaging data analysis based on mutual information estimated via a gaussian copula , 2016, bioRxiv.

[41]  Jonathan Z Simon,et al.  The encoding of auditory objects in auditory cortex: insights from magnetoencephalography. , 2015, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[42]  J. Ziegler,et al.  Speech-perception-in-noise deficits in dyslexia. , 2009, Developmental science.

[43]  Jamie Zibulsky,et al.  Wechsler Nonverbal Scale of Ability , 2014 .

[44]  S. Taulu,et al.  Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements , 2006, Physics in medicine and biology.

[45]  S. Crewther,et al.  The Contribution of Phonological Awareness to Reading Fluency and Its Individual Sub-skills in Readers Aged 9- to 12-years , 2017, Front. Psychol..

[46]  F. Ramus The neural basis of reading acquisition , 2003 .

[47]  Fiona J. Duff,et al.  Evidence for semantic involvement in regular and exception word reading in emergent readers of English. , 2016, Journal of experimental child psychology.

[48]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[49]  D. Share,et al.  Sources of Individual Differences in Reading Acquisition. , 1984 .

[50]  Charles Hulme,et al.  The Foundations of Spelling Ability: Evidence from a 3-Year Longitudinal Study , 2001 .

[51]  Lauri Parkkonen,et al.  Recording epileptic activity with MEG in a light-weight magnetic shield , 2008, Epilepsy Research.

[52]  L. Siegel,et al.  Development of phonological and orthographic processing in reading aloud, in silent reading, and in spelling: a four-year longitudinal study. , 2003, Journal of experimental child psychology.

[53]  Jan Wouters,et al.  Speech Intelligibility Predicted from Neural Entrainment of the Speech Envelope , 2018, bioRxiv.

[54]  Edmund C. Lalor,et al.  Electrophysiological Correlates of Semantic Dissimilarity Reflect the Comprehension of Natural, Narrative Speech , 2017, Current Biology.

[55]  Guinevere F. Eden,et al.  Meta-Analysis of the Functional Neuroanatomy of Single-Word Reading: Method and Validation , 2002, NeuroImage.

[56]  S. Nittrouer From Ear to Cortex: A Perspective on What Clinicians Need to Understand About Speech Perception and Language Processing. , 2002, Language, speech, and hearing services in schools.

[57]  V. Mann,et al.  Speech perception and memory coding in relation to reading ability. , 1983, Journal of experimental child psychology.

[58]  E Ahissar,et al.  Speech comprehension is correlated with temporal response patterns recorded from auditory cortex , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[59]  D. Shankweiler,et al.  Phonetic coding and order memory in relation to reading proficiency: A comparison of short-term memory for temporal and spatial order information , 1983, Applied Psycholinguistics.

[60]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[61]  S. Dehaene,et al.  The unique role of the visual word form area in reading , 2011, Trends in Cognitive Sciences.

[62]  Lars Meyer,et al.  Linguistic Bias Modulates Interpretation of Speech via Neural Delta-Band Oscillations , 2016, Cerebral cortex.

[63]  Cas Smits,et al.  The influence of lexical-access ability and vocabulary knowledge on measures of speech recognition in noise , 2016, International journal of audiology.

[64]  K. Eklund,et al.  Infant brain responses associated with reading-related skills before school and at school age , 2012, Neurophysiologie Clinique/Clinical Neurophysiology.

[65]  Bruno Fimm,et al.  A test battery for attentional performance , 2002 .

[66]  J. Stockman Aircraft and Road Traffic Noise and Children's Cognition and Health: A Cross-National Study , 2007 .

[67]  J Head,et al.  Aircraft and road traffic noise and children's cognition and health: a cross-national study , 2005, The Lancet.

[68]  Jan Wouters,et al.  Speech intelligibility predicted from neural entrainment of the speech envelope , 2018 .

[69]  George K. Georgiou,et al.  Why is rapid automatized naming related to reading? , 2013, Journal of experimental child psychology.

[70]  John J. Foxe,et al.  Attentional Selection in a Cocktail Party Environment Can Be Decoded from Single-Trial EEG. , 2015, Cerebral cortex.

[71]  Charles Hulme,et al.  Rapid Automatized Naming (RAN) Taps a Mechanism That Places Constraints on the Development of Early Reading Fluency , 2009, Psychological science.

[72]  Paul W. B. Atkins,et al.  Models of reading aloud: Dual-route and parallel-distributed-processing approaches. , 1993 .

[73]  Marco Zorzi,et al.  Nested incremental modeling in the development of computational theories: the CDP+ model of reading aloud. , 2007, Psychological review.

[74]  F. Manis,et al.  Naming Speed, Phonological Awareness, and Orthographic Knowledge in Second Graders , 2000, Journal of learning disabilities.

[75]  Jon Barker,et al.  The foreign language cocktail party problem: Energetic and informational masking effects in non-native speech perception. , 2008, The Journal of the Acoustical Society of America.

[76]  Torsten Dau,et al.  Noise-robust cortical tracking of attended speech in real-world acoustic scenes , 2017, NeuroImage.

[77]  C. Schroeder,et al.  Attention modulates ‘speech-tracking’ at a cocktail party , 2012, Trends in Cognitive Sciences.

[78]  Bruno L. Giordano,et al.  Quantitatively comparing predictive models with the Partial Information Decomposition , 2019 .

[79]  Jack A. Naglieri,et al.  Wechsler Nonverbal Scale of Ability (WNV). , 2009 .

[80]  J. Wouters,et al.  Modelling relations between sensory processing, speech perception, orthographic and phonological ability, and literacy achievement , 2008, Brain and Language.

[81]  Stefan Samuelsson,et al.  The impact of environmental factors on components of reading and dyslexia , 2003 .

[82]  Gregor Thut,et al.  Lip movements entrain the observers’ low-frequency brain oscillations to facilitate speech intelligibility , 2016, eLife.

[83]  Nina Kraus,et al.  Individual differences in speech-in-noise perception parallel neural speech processing and attention in preschoolers , 2017, Hearing Research.

[84]  Michael J. Crosse,et al.  Eye Can Hear Clearly Now: Inverse Effectiveness in Natural Audiovisual Speech Processing Relies on Long-Term Crossmodal Temporal Integration , 2016, The Journal of Neuroscience.

[85]  Riitta Hari,et al.  Audiovisual Integration of Letters in the Human Brain , 2000, Neuron.

[86]  Riitta Hari,et al.  Left Superior Temporal Gyrus Is Coupled to Attended Speech in a Cocktail-Party Auditory Scene , 2016, The Journal of Neuroscience.

[87]  S. Scott,et al.  Intact But Less Accessible Phonetic Representations in Adults with Dyslexia , 2013, Science.

[88]  John S. Bradley,et al.  Revisiting Speech Interference in Classrooms:Revisando la interferencia en el habla dentro del salón de clases , 2001 .

[89]  Stefan Debener,et al.  The Right Temporoparietal Junction Supports Speech Tracking During Selective Listening: Evidence from Concurrent EEG-fMRI , 2017, The Journal of Neuroscience.

[90]  Joachim Gross,et al.  Phase-Locked Responses to Speech in Human Auditory Cortex are Enhanced During Comprehension , 2012, Cerebral cortex.

[91]  D. Poeppel,et al.  Mechanisms Underlying Selective Neuronal Tracking of Attended Speech at a “Cocktail Party” , 2013, Neuron.

[92]  Mireille Bastien-Toniazzo,et al.  Audio-Visual Perception and Integration in Developmental Dyslexia: An Exploratory Study Using the McGurk Effect , 2010 .

[93]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[94]  Irwin Pollack,et al.  Auditory informational masking , 1975 .

[95]  D. Allport,et al.  Components of the Mental Lexicon , 1981 .

[96]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[97]  Jonathan Z. Simon,et al.  Adaptive Temporal Encoding Leads to a Background-Insensitive Cortical Representation of Speech , 2013, The Journal of Neuroscience.

[98]  J. Simon,et al.  Emergence of neural encoding of auditory objects while listening to competing speakers , 2012, Proceedings of the National Academy of Sciences.

[99]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[100]  V. Mann,et al.  Using auditory-visual speech to probe the basis of noise-impaired consonant-vowel perception in dyslexia and auditory neuropathy. , 2005, The Journal of the Acoustical Society of America.

[101]  B. Kollmeier,et al.  Age-Related Differences in Lexical Access Relate to Speech Recognition in Noise , 2016, Front. Psychol..

[102]  N. Kraus,et al.  Auditory Processing in Noise: A Preschool Biomarker for Literacy , 2015, PLoS biology.

[103]  P. Deltenre,et al.  Informational masking of speech in dyslexic children. , 2015, The Journal of the Acoustical Society of America.

[104]  Nicola Molinaro,et al.  Hearing through lip-reading: the brain synthesizes features of absent speech , 2018, bioRxiv.

[105]  John J. Foxe,et al.  Neural responses to uninterrupted natural speech can be extracted with precise temporal resolution , 2010, The European journal of neuroscience.

[106]  Elisa Leonardelli,et al.  A Visual Cortical Network for Deriving Phonological Information from Intelligible Lip Movements , 2018, Current Biology.

[107]  P. Stelmachowicz,et al.  Relationship Between Speech Perception in Noise and Phonological Awareness Skills for Children With Normal Hearing , 2010, Ear and hearing.

[108]  Miguel Torres,et al.  Cancer predisposition: Searching for early events , 2015, Nature Reviews Cancer.

[109]  Manuel Carreiras,et al.  Out‐of‐synchrony speech entrainment in developmental dyslexia , 2016, Human brain mapping.

[110]  Robin A. A. Ince,et al.  Neural processing of the same, behaviourally relevant face features is delayed by 40 ms in healthy ageing , 2018, bioRxiv.

[111]  S. Petersen,et al.  Neuroimaging studies of word reading. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[112]  David Poeppel,et al.  The effects of selective attention and speech acoustics on neural speech-tracking in a multi-talker scene , 2015, Cortex.

[113]  H. Wimmer,et al.  The double-deficit hypothesis and difficulties in learning to read a regular orthography. , 2000 .

[114]  Aircraft and road traffic noise and children's cognition , 2005, The Lancet.

[115]  Wouter A Dreschler,et al.  Release from informational masking by time reversal of native and non-native interfering speech. , 2005, The Journal of the Acoustical Society of America.

[116]  S. Taulu,et al.  Applications of the signal space separation method , 2005, IEEE Transactions on Signal Processing.

[117]  Joachim Gross,et al.  Simple Acoustic Features Can Explain Phoneme-Based Predictions of Cortical Responses to Speech , 2019, Current Biology.

[118]  Ramesh Srinivasan,et al.  Suppression of competing speech through entrainment of cortical oscillations. , 2013, Journal of neurophysiology.

[119]  S. Trehub,et al.  Children's perception of speech in multitalker babble. , 2000, The Journal of the Acoustical Society of America.

[120]  J. Simon,et al.  Cortical entrainment to continuous speech: functional roles and interpretations , 2014, Front. Hum. Neurosci..

[121]  J. Vroomen,et al.  Lipread-induced phonetic recalibration in dyslexia. , 2012, Acta psychologica.

[122]  George K. Georgiou,et al.  Phonological Awareness and Rapid Automatized Naming as Longitudinal Predictors of Reading in Five Alphabetic Orthographies with Varying Degrees of Consistency , 2018, Scientific Studies of Reading.

[123]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[124]  J. Vroomen,et al.  A Selective Deficit in Phonetic Recalibration by Text in Developmental Dyslexia , 2018, Front. Psychol..

[125]  J. S. Bradley,et al.  Revisiting speech interference in classrooms. , 2001, Audiology : official organ of the International Society of Audiology.

[126]  J. Vroomen,et al.  Audio-visual speech in noise perception in dyslexia. , 2018, Developmental science.

[127]  Fanny Meunier,et al.  Phonetic and lexical interferences in informational masking during speech-in-speech comprehension , 2007, Speech Commun..

[128]  Alan D. Baddeley,et al.  Phonological working memory: A critical building block for reading development and vocabulary acquisition? , 1993 .

[129]  Jonas Obleser,et al.  Transcranial alternating current stimulation with speech envelopes modulates speech comprehension , 2018, NeuroImage.

[130]  H. Wimmer,et al.  Poor Reading: A Deficit in Skill-Automatization or a Phonological Deficit? , 1998 .

[131]  Nicola Molinaro,et al.  Delta(but not theta)‐band cortical entrainment involves speech‐specific processing , 2018, The European journal of neuroscience.

[132]  Edmund C. Lalor,et al.  The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli , 2016, Front. Hum. Neurosci..

[133]  Donald Shankweiler,et al.  The speech code and learning to read. , 1979 .

[134]  Nina Kraus,et al.  Sensory-cognitive interaction in the neural encoding of speech in noise: a review. , 2010, Journal of the American Academy of Audiology.

[135]  M Coltheart,et al.  DRC: a dual route cascaded model of visual word recognition and reading aloud. , 2001, Psychological review.

[136]  Nicola Molinaro,et al.  Lip-Reading Enables the Brain to Synthesize Auditory Features of Unknown Silent Speech , 2019, The Journal of Neuroscience.

[137]  P. Zimmermann,et al.  Applied Neuropsychology of Attention. Theory, Diagnosis and Rehabilitation. , 2002 .

[138]  J. Schwartz,et al.  Seeing to hear better: evidence for early audio-visual interactions in speech identification , 2004, Cognition.

[139]  Leo Blomert,et al.  The neural signature of orthographic–phonological binding in successful and failing reading development , 2011, NeuroImage.

[140]  Edmund C Lalor,et al.  The cortical representation of the speech envelope is earlier for audiovisual speech than audio speech. , 2014, Journal of neurophysiology.