Frontotemporal activation differs between perception of simulated cochlear implant speech and speech in background noise: An image-based fNIRS study

In this study we used functional near-infrared spectroscopy (fNIRS) to investigate neural responses in normal-hearing adults as a function of speech recognition accuracy, intelligibility of the speech stimulus, and the manner in which speech is distorted. Participants listened to sentences and reported aloud what they heard. Speech quality was distorted artificially by vocoding (simulated cochlear implant speech) or naturally by adding background noise. Each type of distortion included high and low-intelligibility conditions. Sentences in quiet were used as baseline comparison. fNIRS data were analyzed using a newly developed image reconstruction approach. First, elevated cortical responses in the middle temporal gyrus (MTG) and middle frontal gyrus (MFG) were associated with speech recognition during the low-intelligibility conditions. Second, activation in the MTG was associated with recognition of vocoded speech with low intelligibility, whereas MFG activity was largely driven by recognition of speech in background noise, suggesting that the cortical response varies as a function of distortion type. Lastly, an accuracy effect in the MFG demonstrated significantly higher activation during correct perception relative to incorrect perception of speech. These results suggest that normal-hearing adults (i.e., untrained listeners of vocoded stimuli) do not exploit the same attentional mechanisms of the frontal cortex used to resolve naturally degraded speech and may instead rely on segmental and phonetic analyses in the temporal lobe to discriminate vocoded speech.

[1]  D. Hartley,et al.  The use of functional near-infrared spectroscopy for measuring cortical reorganisation in cochlear implant users: A possible predictor of variable speech outcomes? , 2015, Cochlear implants international.

[2]  R. Litovsky,et al.  Recognition of vocoded words and sentences in quiet and multi-talker babble with children and adults , 2020, PloS one.

[3]  T. Parrish,et al.  Cortical mechanisms of speech perception in noise. , 2008, Journal of speech, language, and hearing research : JSLHR.

[4]  A. Zekveld,et al.  Cognitive processing load across a wide range of listening conditions: insights from pupillometry. , 2014, Psychophysiology.

[5]  Abraham Z. Snyder,et al.  A quantitative spatial comparison of high-density diffuse optical tomography and fMRI cortical mapping , 2012, NeuroImage.

[6]  Aaron T. Buss,et al.  Validating an image-based fNIRS approach with fMRI and a working memory task , 2017, NeuroImage.

[7]  M. Beauchamp,et al.  Auditory cortex activation to natural speech and simulated cochlear implant speech measured with functional near-infrared spectroscopy , 2014, Hearing Research.

[8]  R. Litovsky,et al.  Cortical Speech Processing in Postlingually Deaf Adult Cochlear Implant Users, as Revealed by Functional Near-Infrared Spectroscopy , 2018, Trends in hearing.

[9]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[10]  R. Poldrack,et al.  Relations between the Neural Bases of Dynamic Auditory Processing and Phonological Processing: Evidence from fMRI , 2001, Journal of Cognitive Neuroscience.

[11]  Stuart Rosen,et al.  A positron emission tomography study of the neural basis of informational and energetic masking effects in speech perception. , 2004, The Journal of the Acoustical Society of America.

[12]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[13]  D. Hartley,et al.  Brain activity underlying the recovery of meaning from degraded speech: A functional near-infrared spectroscopy (fNIRS) study , 2017, Hearing Research.

[14]  Ferath Kherif,et al.  Does Semantic Context Benefit Speech Understanding through “Top–Down” Processes? Evidence from Time-resolved Sparse fMRI , 2011, Journal of Cognitive Neuroscience.

[15]  David A Boas,et al.  Monte Carlo simulation of photon migration in 3D turbid media accelerated by graphics processing units. , 2009, Optics express.

[16]  P. Matthews,et al.  Defining a left-lateralized response specific to intelligible speech using fMRI. , 2003, Cerebral cortex.

[17]  Alexander L. Francis,et al.  Autonomic Nervous System Responses During Perception of Masked Speech may Reflect Constructs other than Subjective Listening Effort , 2016, Front. Psychol..

[18]  A. Samuel,et al.  Perceptual learning for speech , 2009, Attention, perception & psychophysics.

[19]  Ellen Leibenluft,et al.  Applications of multivariate modeling to neuroimaging group analysis: A comprehensive alternative to univariate general linear model , 2014, NeuroImage.

[20]  Y. Samson,et al.  Lateralization of Speech and Auditory Temporal Processing , 1998, Journal of Cognitive Neuroscience.

[21]  D. Gow The cortical organization of lexical knowledge: A dual lexicon model of spoken language processing , 2012, Brain and Language.

[22]  E. Formisano,et al.  Neural correlates of verbal feedback processing: An fMRI study employing overt speech , 2007, Human brain mapping.

[23]  P Sterzer,et al.  Contributions of sensory input, auditory search and verbal comprehension to cortical activity during speech processing. , 2004, Cerebral cortex.

[24]  M. Kathleen Pichora-Fuller,et al.  How Social Psychological Factors May Modulate Auditory and Cognitive Functioning During Listening , 2016, Ear and hearing.

[25]  D. Başkent,et al.  Listening effort with cochlear implant simulations. , 2013, Journal of speech, language, and hearing research : JSLHR.

[26]  Matthew H. Davis,et al.  Lexical information drives perceptual learning of distorted speech: evidence from the comprehension of noise-vocoded sentences. , 2005, Journal of experimental psychology. General.

[27]  A. Friederici The brain basis of language processing: from structure to function. , 2011, Physiological reviews.

[28]  Andreas Hein,et al.  Characterizing the Influence of Muscle Activity in fNIRS Brain Activation Measurements , 2016 .

[29]  Judit Gervain,et al.  Efficient Neural Coding in Auditory and Speech Perception , 2019, Trends in Neurosciences.

[30]  Ronny K. Ibrahim,et al.  Objective Assessment of Listening Effort: Coregistration of Pupillometry and EEG , 2017, Trends in hearing.

[31]  Sophie K. Scott,et al.  Semantic versus perceptual interactions in neural processing of speech-in-noise , 2013, NeuroImage.

[32]  Lourdes M Delgado Reyes,et al.  Early adversity in rural India impacts the brain networks underlying visual working memory , 2019, Developmental science.

[33]  Martin Wolf,et al.  End-tidal CO2: An important parameter for a correct interpretation in functional brain studies using speech tasks , 2013, NeuroImage.

[34]  Chris James,et al.  Contralateral Masking in Cochlear Implant Users with Residual Hearing in the Non-Implanted Ear , 2001, Audiology and Neurotology.

[35]  S Nioka,et al.  Comparison of time-resolved and -unresolved measurements of deoxyhemoglobin in brain. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Jonas Obleser,et al.  Bilateral Speech Comprehension Reflects Differential Sensitivity to Spectral and Temporal Features , 2008, The Journal of Neuroscience.

[37]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[38]  John G. Kerns,et al.  Anterior cingulate and prefrontal cortex activity in an FMRI study of trial-to-trial adjustments on the Simon task , 2006, NeuroImage.

[39]  P. Vuust,et al.  Cortical Plasticity after Cochlear Implantation , 2013, Neural plasticity.

[40]  Joanna H. Lowenstein,et al.  Word Recognition Variability With Cochlear Implants: “Perceptual Attention” Versus “Auditory Sensitivity” , 2016, Ear and hearing.

[41]  K. R. Ridderinkhof,et al.  Medial frontal cortex and response conflict: Evidence from human intracranial EEG and medial frontal cortex lesion , 2008, Brain Research.

[42]  S. Petersen,et al.  A dual-networks architecture of top-down control , 2008, Trends in Cognitive Sciences.

[43]  Matthew K. Leonard,et al.  Perceptual restoration of masked speech in human cortex , 2016, Nature Communications.

[44]  S. Debener,et al.  Rapid bilateral improvement in auditory cortex activity in postlingually deafened adults following cochlear implantation , 2015, Clinical Neurophysiology.

[45]  B. J. Casey,et al.  Activation of the prefrontal cortex in a nonspatial working memory task with functional MRI , 1994, Human brain mapping.

[46]  D. Calvetti,et al.  Tikhonov regularization and the L-curve for large discrete ill-posed problems , 2000 .

[47]  Joseph P Culver,et al.  High-density diffuse optical tomography for imaging human brain function , 2019, The Review of scientific instruments.

[48]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[49]  Antoine J. Shahin,et al.  Do adults with cochlear implants rely on different acoustic cues for phoneme perception than adults with normal hearing? , 2014, Journal of speech, language, and hearing research : JSLHR.

[50]  S. Scott,et al.  Functional Integration across Brain Regions Improves Speech Perception under Adverse Listening Conditions , 2007, The Journal of Neuroscience.

[51]  J. Obleser,et al.  Expectancy constraints in degraded speech modulate the language comprehension network. , 2010, Cerebral cortex.

[52]  G. Dumont,et al.  Wavelet based motion artifact removal for Functional Near Infrared Spectroscopy , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[53]  K. R. Ridderinkhof,et al.  Neurocognitive mechanisms of cognitive control: The role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning , 2004, Brain and Cognition.

[54]  D. Boas,et al.  HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. , 2009, Applied optics.

[55]  Steve Majerus,et al.  Modulation of brain activity during phonological familiarization , 2005, Brain and Language.

[56]  Mahlega S. Hassanpour,et al.  Mapping distributed brain function and networks with diffuse optical tomography , 2014, Nature Photonics.

[57]  K Shoji,et al.  Functional differentiation of the auditory association area in prelingually deaf subjects. , 2000, Auris, nasus, larynx.

[58]  Edward E. Smith,et al.  A Parametric Study of Prefrontal Cortex Involvement in Human Working Memory , 1996, NeuroImage.

[59]  S. Debener,et al.  Cross-Modal Functional Reorganization of Visual and Auditory Cortex in Adult Cochlear Implant Users Identified with fNIRS , 2015, Neural plasticity.

[60]  A. Villringer,et al.  Near infrared spectroscopy (NIRS): A new tool to study hemodynamic changes during activation of brain function in human adults , 1993, Neuroscience Letters.

[61]  Terrin N Tamati,et al.  List Equivalency of PRESTO for the Evaluation of Speech Recognition. , 2015, Journal of the American Academy of Audiology.

[62]  Timothy E. J. Behrens,et al.  Functional organization of the medial frontal cortex , 2007, Current Opinion in Neurobiology.

[63]  Susan Teubner-Rhodes,et al.  Is Listening in Noise Worth It? The Neurobiology of Speech Recognition in Challenging Listening Conditions. , 2016, Ear and hearing.

[64]  M. Cooke,et al.  Recognizing speech under a processing load: Dissociating energetic from informational factors , 2009, Cognitive Psychology.

[65]  Stefanie E. Kuchinsky,et al.  The Cingulo-Opercular Network Provides Word-Recognition Benefit , 2013, The Journal of Neuroscience.

[66]  D. Poeppel,et al.  Speech perception at the interface of neurobiology and linguistics , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[67]  Adam T. Eggebrecht,et al.  NeuroDOT: an extensible Matlab toolbox for streamlined optical functional mapping , 2019, European Conference on Biomedical Optics.

[68]  G. Christopher Stecker,et al.  Human evoked cortical activity to signal-to-noise ratio and absolute signal level , 2009, Hearing Research.

[69]  M Wolf,et al.  The confounding effect of systemic physiology on the hemodynamic response in newborns. , 2012, Advances in experimental medicine and biology.

[70]  Patryk A. Laurent,et al.  Evidence for Cerebellar Contributions to Adaptive Plasticity in Speech Perception. , 2015, Cerebral cortex.

[71]  Marc Rousseaux,et al.  Divided and Focused Attention in Patients with Lesion of the Prefrontal Cortex , 1996, Brain and Cognition.

[72]  A. Ehlis,et al.  Influence of muscle activity on brain oxygenation during verbal fluency assessed with functional near-infrared spectroscopy , 2010, Neuroscience.

[73]  J. Davies-Thompson,et al.  Cortical correlates of speech intelligibility measured using functional near-infrared spectroscopy (fNIRS) , 2018, Hearing Research.

[74]  E. Chang,et al.  Human cortical sensorimotor network underlying feedback control of vocal pitch , 2013, Proceedings of the National Academy of Sciences.

[75]  T. Lunner,et al.  The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances , 2013, Front. Syst. Neurosci..

[76]  Abraham Z. Snyder,et al.  Statistical analysis of high density diffuse optical tomography , 2014, NeuroImage.

[77]  D. Boas,et al.  Non-invasive neuroimaging using near-infrared light , 2002, Biological Psychiatry.

[78]  David A. Boas,et al.  Improved recovery of the hemodynamic response in diffuse optical imaging using short optode separations and state-space modeling , 2011, NeuroImage.

[79]  M Wolf,et al.  How to detect and reduce movement artifacts in near-infrared imaging using moving standard deviation and spline interpolation , 2010, Physiological measurement.

[80]  Cécile Issard,et al.  Variability of the hemodynamic response in infants: Influence of experimental design and stimulus complexity , 2018, Developmental Cognitive Neuroscience.

[81]  S. Nagarajan,et al.  Speech target modulates speaking induced suppression in auditory cortex , 2009, BMC Neuroscience.

[82]  Fabrice Wallois,et al.  Effect of confounding variables on hemodynamic response function estimation using averaging and deconvolution analysis: An event-related NIRS study , 2017, NeuroImage.

[83]  Meryem A Yücel,et al.  Motion artifact detection and correction in functional near-infrared spectroscopy: a new hybrid method based on spline interpolation method and Savitzky–Golay filtering , 2018, Neurophotonics.

[84]  A. Savitzky,et al.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures. , 1964 .

[85]  L Whitford,et al.  Factors affecting auditory performance of postlinguistically deaf adults using cochlear implants. , 1996, Audiology & neuro-otology.

[86]  Thomas Lunner,et al.  Impact of SNR, masker type and noise reduction processing on sentence recognition performance and listening effort as indicated by the pupil dilation response , 2018, Hearing Research.

[87]  Marco Pelizzone,et al.  Acoustic to Electric Pitch Comparisons in Cochlear Implant Subjects with Residual Hearing , 2006, Journal of the Association for Research in Otolaryngology.

[88]  Yinjuan Du,et al.  Noise differentially impacts phoneme representations in the auditory and speech motor systems , 2014, Proceedings of the National Academy of Sciences.

[89]  F. Scholkmann,et al.  The effect of inner speech on arterial CO2 and cerebral hemodynamics and oxygenation: a functional NIRS study. , 2013, Advances in experimental medicine and biology.

[90]  Samuel H. Forbes,et al.  Processing pipeline for image reconstructed fNIRS analysis using both MRI templates and individual anatomy , 2021, Neurophotonics.

[91]  A M Dale,et al.  Optimal experimental design for event‐related fMRI , 1999, Human brain mapping.

[92]  A. Villringer,et al.  Illuminating the BOLD signal: combined fMRI-fNIRS studies. , 2006, Magnetic resonance imaging.

[93]  Michael S. Beauchamp,et al.  Cortical Activation Patterns Correlate with Speech Understanding After Cochlear Implantation , 2016, Ear and hearing.

[94]  M. Ferrari,et al.  A brief review on the use of functional near-infrared spectroscopy (fNIRS) for language imaging studies in human newborns and adults , 2012, Brain and Language.

[95]  C. Fiebach,et al.  The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. , 2003, Cerebral cortex.

[96]  S. Soli,et al.  Development of the Hearing in Noise Test for the measurement of speech reception thresholds in quiet and in noise. , 1994, The Journal of the Acoustical Society of America.

[97]  David R Moore,et al.  Neural indices of listening effort in noisy environments , 2019, Scientific Reports.

[98]  Ilias Tachtsidis,et al.  The Effect on Cerebral Tissue Oxygenation Index of Changes in the Concentrations of Inspired Oxygen and End-Tidal Carbon Dioxide in Healthy Adult Volunteers , 2009, Anesthesia and analgesia.

[99]  Masa-aki Sato,et al.  Reduction of global interference of scalp-hemodynamics in functional near-infrared spectroscopy using short distance probes , 2016, NeuroImage.

[100]  Matthew H. Davis,et al.  Neural dissociation in processing noise and accent in spoken language comprehension , 2012, Neuropsychologia.

[101]  A. Beynon,et al.  Pre-, Per- and Postoperative Factors Affecting Performance of Postlinguistically Deaf Adults Using Cochlear Implants: A New Conceptual Model over Time , 2012, PloS one.

[102]  Theodore J Huppert,et al.  Commentary on the statistical properties of noise and its implication on general linear models in functional near-infrared spectroscopy. , 2016, Neurophotonics.

[103]  D. Poeppel,et al.  Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language , 2004, Cognition.

[104]  D. Boas,et al.  Effective scattering coefficient of the cerebral spinal fluid in adult head models for diffuse optical imaging. , 2006, Applied optics.

[105]  Matthew B. Winn,et al.  The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation , 2015, Ear and hearing.

[106]  A. Hielscher,et al.  Three-dimensional optical tomography of hemodynamics in the human head. , 2001, Optics express.

[107]  Matthew H. Davis,et al.  The neural mechanisms of speech comprehension: fMRI studies of semantic ambiguity. , 2005, Cerebral cortex.

[108]  Aaron T. Buss,et al.  Investigating the role of temporal lobe activation in speech perception accuracy with normal hearing adults: An event-related fNIRS study , 2017, Neuropsychologia.

[109]  S. Scott,et al.  The Pathways for Intelligible Speech: Multivariate and Univariate Perspectives , 2013, Cerebral cortex.

[110]  João Ricardo Sato,et al.  Non-neuronal evoked and spontaneous hemodynamic changes in the anterior temporal region of the human head may lead to misinterpretations of functional near-infrared spectroscopy signals , 2017, Neurophotonics.

[111]  Ann-Christine Ehlis,et al.  The Temporal Muscle of the Head Can Cause Artifacts in Optical Imaging Studies with Functional Near-Infrared Spectroscopy , 2017, Front. Hum. Neurosci..

[112]  Paul A. Taylor,et al.  FMRI Clustering in AFNI: False-Positive Rates Redux , 2017, Brain Connect..

[113]  E. Toppila,et al.  The effect of different noise types on the speech and non-speech elicited mismatch negativity , 2005, Hearing Research.

[114]  Signy Sheldon,et al.  Priming and sentence context support listening to noise-vocoded speech by younger and older adults. , 2008, The Journal of the Acoustical Society of America.

[115]  J. Peelle Listening Effort: How the Cognitive Consequences of Acoustic Challenge Are Reflected in Brain and Behavior , 2017, Ear and hearing.

[116]  Matthew H. Davis,et al.  Hierarchical Processing in Spoken Language Comprehension , 2003, The Journal of Neuroscience.

[117]  S. Scott,et al.  Inferior Frontal Gyrus Activation Predicts Individual Differences in Perceptual Learning of Cochlear-Implant Simulations , 2010, The Journal of Neuroscience.

[118]  A. Beynon,et al.  Factors Affecting Auditory Performance of Postlinguistically Deaf Adults Using Cochlear Implants: An Update with 2251 Patients , 2012, Audiology and Neurotology.

[119]  Ingrid S. Johnsrude,et al.  The eye as a window to the listening brain: Neural correlates of pupil size as a measure of cognitive listening load , 2014, NeuroImage.

[120]  Matthew H. Davis,et al.  Speech recognition in adverse conditions: A review , 2012 .

[121]  P. Marquet,et al.  In vivo local determination of tissue optical properties: applications to human brain. , 1999, Applied optics.

[122]  Jonathan D. Cohen,et al.  The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function , 2013, Neuron.

[123]  Thomas J. Grabowski,et al.  The Left Posterior Superior Temporal Gyrus Participates Specifically in Accessing Lexical Phonology , 2008, Journal of Cognitive Neuroscience.

[124]  R. Buxton,et al.  Dynamics of blood flow and oxygenation changes during brain activation: The balloon model , 1998, Magnetic resonance in medicine.

[125]  David A. Boas,et al.  Motion artifacts in functional near-infrared spectroscopy: A comparison of motion correction techniques applied to real cognitive data , 2014, NeuroImage.

[126]  Matthew H. Davis,et al.  Brain regions recruited for the effortful comprehension of noise-vocoded words , 2012 .

[127]  D. Levitin,et al.  Functional near-infrared spectroscopy for neuroimaging in cochlear implant recipients , 2016, Hearing Research.

[128]  N Marangos,et al.  COCHLEAR IMPLANTS , 1976, The Lancet.

[129]  Jemett L. Desmond,et al.  Semantic encoding and retrieval in the left inferior prefrontal cortex: a functional MRI study of task difficulty and process specificity , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[130]  L. Pylkkänen,et al.  Adaptation to mis-pronounced speech: evidence for a prefrontal-cortex repair mechanism , 2021, Scientific reports.

[131]  Matthew H. Davis,et al.  Effortful Listening: The Processing of Degraded Speech Depends Critically on Attention , 2012, The Journal of Neuroscience.

[132]  J. Schnupp Auditory Filters, Features, and Redundant Representations , 2006, Neuron.