Effects of Additional Low-Pass–Filtered Speech on Listening Effort for Noise-Band–Vocoded Speech in Quiet and in Noise

Objectives: Residual acoustic hearing in electric–acoustic stimulation (EAS) can benefit cochlear implant (CI) users in increased sound quality, speech intelligibility, and improved tolerance to noise. The goal of this study was to investigate whether the low-pass–filtered acoustic speech in simulated EAS can provide the additional benefit of reducing listening effort for the spectrotemporally degraded signal of noise-band–vocoded speech. Design: Listening effort was investigated using a dual-task paradigm as a behavioral measure, and the NASA Task Load indeX as a subjective self-report measure. The primary task of the dual-task paradigm was identification of sentences presented in three experiments at three fixed intelligibility levels: at near-ceiling, 50%, and 79% intelligibility, achieved by manipulating the presence and level of speech-shaped noise in the background. Listening effort for the primary intelligibility task was reflected in the performance on the secondary, visual response time task. Experimental speech processing conditions included monaural or binaural vocoder, with added low-pass–filtered speech (to simulate EAS) or without (to simulate CI). Results: In Experiment 1, in quiet with intelligibility near-ceiling, additional low-pass–filtered speech reduced listening effort compared with binaural vocoder, in line with our expectations, although not compared with monaural vocoder. In Experiments 2 and 3, for speech in noise, added low-pass–filtered speech allowed the desired intelligibility levels to be reached at less favorable speech-to-noise ratios, as expected. It is interesting that this came without the cost of increased listening effort usually associated with poor speech-to-noise ratios; at 50% intelligibility, even a reduction in listening effort on top of the increased tolerance to noise was observed. The NASA Task Load indeX did not capture these differences. Conclusions: The dual-task results provide partial evidence for a potential decrease in listening effort as a result of adding low-frequency acoustic speech to noise-band–vocoded speech. Whether these findings translate to CI users with residual acoustic hearing will need to be addressed in future research because the quality and frequency range of low-frequency acoustic sound available to listeners with hearing loss may differ from our idealized simulations, and additional factors, such as advanced age and varying etiology, may also play a role.

[1]  W. Noble,et al.  Unilateral and bilateral cochlear implants and the implant-plus-hearing-aid profile: Comparing self-assessed and measured abilities , 2008, International journal of audiology.

[2]  Samantha Gustafson,et al.  Listening Effort and Perceived Clarity for Normal-Hearing Children With the Use of Digital Noise Reduction , 2014, Ear and hearing.

[3]  U. Lemke,et al.  Cognitive Load and Listening Effort: Concepts and Age-Related Considerations , 2016, Ear and hearing.

[4]  Deniz Başkent,et al.  Effect of Speech Degradation on Top-Down Repair: Phonemic Restoration with Simulations of Cochlear Implants and Combined Electric–Acoustic Stimulation , 2012, Journal of the Association for Research in Otolaryngology.

[5]  Christopher J. Plack,et al.  Listening effort at signal-to-noise ratios that are typical of the school classroom , 2010, International journal of audiology.

[6]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[7]  John F Culling,et al.  The role of fundamental frequency contours in the perception of speech against interfering speech. , 2005, The Journal of the Acoustical Society of America.

[8]  Jan Kiefer,et al.  Combined Electric and Acoustic Stimulation of the Auditory System: Results of a Clinical Study , 2005, Audiology and Neurotology.

[9]  Deniz Başkent,et al.  Recognition of temporally interrupted and spectrally degraded sentences with additional unprocessed low-frequency speech , 2010, Hearing Research.

[10]  Bruce J Gantz,et al.  Multicenter clinical trial of the Nucleus Hybrid S8 cochlear implant: Final outcomes , 2016, The Laryngoscope.

[11]  P. Toffanin,et al.  The Timing and Effort of Lexical Access in Natural and Degraded Speech , 2016, Front. Psychol..

[12]  T. Lenarz,et al.  Impact of Low-Frequency Hearing , 2009, Audiology and Neurotology.

[13]  T Houtgast,et al.  Method for the selection of sentence materials for efficient measurement of the speech reception threshold. , 1999, The Journal of the Acoustical Society of America.

[14]  Fan-Gang Zeng,et al.  Bimodal Hearing Benefit for Speech Recognition with Competing Voice in Cochlear Implant Subject with Normal Hearing in Contralateral Ear , 2010, Ear and hearing.

[15]  R. Sims,et al.  Comparison of Psychophysiological and Dual-Task Measures of Listening Effort. , 2015, Journal of speech, language, and hearing research : JSLHR.

[16]  D. Downs Effects of hearing and use on speech discrimination and listening effort. , 1982, Journal of Speech and Hearing Disorders.

[17]  D. D. Greenwood A cochlear frequency-position function for several species--29 years later. , 1990, The Journal of the Acoustical Society of America.

[18]  R. Shannon,et al.  Speech recognition in noise as a function of the number of spectral channels: comparison of acoustic hearing and cochlear implants. , 2001, The Journal of the Acoustical Society of America.

[19]  H Dudley,et al.  The Automatic Synthesis of Speech. , 1939, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Bruce J Gantz,et al.  Optimizing the Combination of Acoustic and Electric Hearing in the Implanted Ear , 2013, Ear and hearing.

[21]  R. Plomp A signal-to-noise ratio model for the speech-reception threshold of the hearing impaired. , 1986, Journal of speech and hearing research.

[22]  Matthew H. Davis,et al.  Effortful Listening: The Processing of Degraded Speech Depends Critically on Attention , 2012, The Journal of Neuroscience.

[23]  Terrin N. Tamati,et al.  Perception and psychoacoustics of speech in cochlear implant users , 2016 .

[24]  Karyn L Galvin,et al.  Measuring listening effort expended by adolescents and young adults with unilateral or bilateral cochlear implants or normal hearing , 2013, Cochlear implants international.

[25]  Jean-Pierre Gagné,et al.  Older adults expend more listening effort than young adults recognizing speech in noise. , 2011, Journal of speech, language, and hearing research : JSLHR.

[26]  Ying-Yee Kong,et al.  Improved speech recognition in noise in simulated binaurally combined acoustic and electric stimulation. , 2007, The Journal of the Acoustical Society of America.

[27]  Christopher A Brown,et al.  Low-frequency speech cues and simulated electric-acoustic hearing. , 2009, The Journal of the Acoustical Society of America.

[28]  Bruce J. Gantz,et al.  Benefits seen in acoustic hearing + electric stimulation in same ear , 2005 .

[29]  Michael K. Qin,et al.  Effects of introducing unprocessed low-frequency information on the reception of envelope-vocoder processed speech. , 2006, The Journal of the Acoustical Society of America.

[30]  Mary Florentine,et al.  Binaural loudness summation for speech presented via earphones and loudspeaker with and without visual cues. , 2012, The Journal of the Acoustical Society of America.

[31]  Jean-Pierre Gagné,et al.  Older adults expend more listening effort than young adults recognizing audiovisual speech in noise , 2011, International journal of audiology.

[32]  Deniz Başkent,et al.  Perceptual Learning of Interrupted Speech , 2013, PloS one.

[33]  D. Başkent,et al.  Listening effort with cochlear implant simulations. , 2013, Journal of speech, language, and hearing research : JSLHR.

[34]  S Gatehouse,et al.  The role of non-auditory factors in measured and self-reported disability. , 1990, Acta oto-laryngologica. Supplementum.

[35]  A. Zekveld,et al.  Pupil Response as an Indication of Effortful Listening: The Influence of Sentence Intelligibility , 2010, Ear and hearing.

[36]  U. Lemke,et al.  Behavioral Assessment of Listening Effort Using a Dual-Task Paradigm , 2017, Trends in hearing.

[37]  J. Rönnberg Cognition in the hearing impaired and deaf as a bridge between signal and dialogue: a framework and a model , 2003, International journal of audiology.

[38]  Mary Florentine,et al.  Binaural Loudness Summation for Speech and Tones Presented via Earphones and Loudspeakers , 2009, Ear and hearing.

[39]  Michael F Dorman,et al.  Information From the Voice Fundamental Frequency (F0) Region Accounts for the Majority of the Benefit When Acoustic Stimulation Is Added to Electric Stimulation , 2010, Ear and hearing.

[40]  A. Cutler,et al.  Cross-language differences in cue use for speech segmentation. , 2009, The Journal of the Acoustical Society of America.

[41]  D. Kahneman,et al.  Attention and Effort , 1973 .

[42]  Deniz Başkent,et al.  The discrimination of voice cues in simulations of bimodal electro-acoustic cochlear-implant hearing. , 2018, The Journal of the Acoustical Society of America.

[43]  Candace Bourland Hick,et al.  Listening effort and fatigue in school-age children with and without hearing loss. , 2002, Journal of speech, language, and hearing research : JSLHR.

[44]  Yu-Hsiang Wu,et al.  Psychometric Functions of Dual-Task Paradigms for Measuring Listening Effort , 2016, Ear and hearing.

[45]  Jerker Rönnberg,et al.  Working memory compensates for hearing related phonological processing deficit. , 2013, Journal of communication disorders.

[46]  C von Ilberg,et al.  Electric-acoustic stimulation of the auditory system. New technology for severe hearing loss. , 1999, ORL; journal for oto-rhino-laryngology and its related specialties.

[47]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[48]  Michael F Dorman,et al.  Cochlear Implantation With Hearing Preservation Yields Significant Benefit for Speech Recognition in Complex Listening Environments , 2013, Ear and hearing.

[49]  A Wingfield,et al.  Speech-processing capacity in young and older adults: a dual-task study. , 1991, Psychology and aging.

[50]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[51]  T. Lunner,et al.  Cognition counts: A working memory system for ease of language understanding (ELU) , 2008, International journal of audiology.

[52]  Tina M. Grieco-Calub,et al.  Age-Related Differences in Listening Effort During Degraded Speech Recognition , 2017, Ear and hearing.

[53]  Matthew B. Winn,et al.  The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation , 2015, Ear and hearing.

[54]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[55]  H. Levitt Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[56]  Fan-Gang Zeng,et al.  Speech and melody recognition in binaurally combined acoustic and electric hearing. , 2005, The Journal of the Acoustical Society of America.

[57]  Philipos C. Loizou,et al.  Acoustic Simulations of Combined Electric and Acoustic Hearing (EAS) , 2005, Ear and hearing.

[58]  A. Oxenham Pitch Perception and Auditory Stream Segregation: Implications for Hearing Loss and Cochlear Implants , 2008, Trends in amplification.

[59]  A. Cutler,et al.  Explaining cross-linguistic differences in effects of lexical stress on spoken-word recognition , 2006, Speech Prosody 2006.

[60]  R. H. Baayen,et al.  The CELEX Lexical Database (CD-ROM) , 1996 .

[61]  A Wingfield,et al.  Cognitive factors in auditory performance: context, speed of processing, and constraints of memory. , 1996, Journal of the American Academy of Audiology.

[62]  T. Lunner,et al.  The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances , 2013, Front. Syst. Neurosci..

[63]  Michael F Dorman,et al.  Frequency Overlap Between Electric and Acoustic Stimulation and Speech-Perception Benefit in Patients with Combined Electric and Acoustic Stimulation , 2010, Ear and hearing.

[64]  Ilona Anderson,et al.  Electric acoustic stimulation of the auditory system: results of a multi-centre investigation , 2008, Acta oto-laryngologica.

[65]  Jean-Pierre Gagné,et al.  Evaluating the effort expended to understand speech in noise using a dual-task paradigm: the effects of providing visual speech cues. , 2010, Journal of speech, language, and hearing research : JSLHR.

[66]  J. F. Feuerstein,et al.  Monaural versus binaural hearing: ease of listening, word recognition, and attentional effort. , 1992, Ear and hearing.

[67]  M. Daneman,et al.  How young and old adults listen to and remember speech in noise. , 1995, The Journal of the Acoustical Society of America.

[68]  A. Neel,et al.  Effects of loud and amplified speech on sentence and word intelligibility in Parkinson disease. , 2009, Journal of speech, language, and hearing research : JSLHR.

[69]  B. Papsin,et al.  Binaural Fusion and Listening Effort in Children Who Use Bilateral Cochlear Implants: A Psychoacoustic and Pupillometric Study , 2015, PloS one.

[70]  Sridhar Kalluri,et al.  Objective measures of listening effort: effects of background noise and noise reduction. , 2009, Journal of speech, language, and hearing research : JSLHR.

[71]  D. G. Payne,et al.  Effects of Speech Intelligibility Level on Concurrent Visual Task Performance , 1994, Human factors.

[72]  Michael F Dorman,et al.  Combining acoustic and electric stimulation in the service of speech recognition , 2010, International journal of audiology.