Impact of Background Noise and Sentence Complexity on Processing Demands during Sentence Comprehension

Speech comprehension in adverse listening conditions can be effortful even when speech is fully intelligible. Acoustical distortions typically make speech comprehension more effortful, but effort also depends on linguistic aspects of the speech signal, such as its syntactic complexity. In the present study, pupil dilations, and subjective effort ratings were recorded in 20 normal-hearing participants while performing a sentence comprehension task. The sentences were either syntactically simple (subject-first sentence structure) or complex (object-first sentence structure) and were presented in two levels of background noise both corresponding to high intelligibility. A digit span and a reading span test were used to assess individual differences in the participants’ working memory capacity (WMC). The results showed that the subjectively rated effort was mostly affected by the noise level and less by syntactic complexity. Conversely, pupil dilations increased with syntactic complexity but only showed a small effect of the noise level. Participants with higher WMC showed increased pupil responses in the higher-level noise condition but rated sentence comprehension as being less effortful compared to participants with lower WMC. Overall, the results demonstrate that pupil dilations and subjectively rated effort represent different aspects of effort. Furthermore, the results indicate that effort can vary in situations with high speech intelligibility.

[1]  S. Kemper,et al.  Life-span changes to adults' language: Effects of memory and genre , 1989, Applied Psycholinguistics.

[2]  Brent Edwards,et al.  The Future of Hearing Aid Technology , 2007, Trends in amplification.

[3]  S Gatehouse,et al.  Response times to speech stimuli as measures of benefit from amplification. , 1990, British journal of audiology.

[4]  A. Zekveld,et al.  Pupil Dilation Uncovers Extra Listening Effort in the Presence of a Single-Talker Masker , 2012, Ear and hearing.

[5]  E. Granholm,et al.  Pupillary responses index cognitive resource limitations. , 1996, Psychophysiology.

[6]  A. Zekveld,et al.  Cognitive Load During Speech Perception in Noise: The Influence of Age, Hearing Loss, and Cognition on the Pupil Response , 2011, Ear and hearing.

[7]  R. Plomp,et al.  Effects of fluctuating noise and interfering speech on the speech-reception threshold for impaired and normal hearing. , 1990, The Journal of the Acoustical Society of America.

[8]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[9]  A. Stewart,et al.  Listening effort and fatigue: What exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group ‘white paper’ , 2014, International journal of audiology.

[10]  G K Poock,et al.  Information processing vs pupil diameter. , 1973, Perceptual and motor skills.

[11]  Karolina Smeds,et al.  Estimation of Signal-to-Noise Ratios in Realistic Sound Scenarios. , 2015, Journal of the American Academy of Audiology.

[12]  T. Lunner,et al.  The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances , 2013, Front. Syst. Neurosci..

[13]  Arthur Wingfield,et al.  Effects of adult aging and hearing loss on comprehension of rapid speech varying in syntactic complexity. , 2006, Journal of the American Academy of Audiology.

[14]  G. Gendolla,et al.  Effort Mobilization when the Self is Involved: Some Lessons from the Cardiovascular System , 2010 .

[15]  G. Buck Assessing Listening , 2001 .

[16]  Birger Kollmeier,et al.  An Eye-Tracking Paradigm for Analyzing the Processing Time of Sentences with Different Linguistic Complexities , 2014, PloS one.

[17]  M. Akeroyd Are individual differences in speech reception related to individual differences in cognitive ability? A survey of twenty experimental studies with normal and hearing-impaired adults , 2008, International journal of audiology.

[18]  Birger Kollmeier,et al.  The role of silent intervals for sentence intelligibility in fluctuating noise in hearing-impaired listeners , 2006, International journal of audiology.

[19]  L. N. Johnson Pupil response. , 1989, The Western journal of medicine.

[20]  I. Johnsrude,et al.  Factors that increase processing demands when listening to speech , 2015 .

[21]  Stefanie E. Kuchinsky,et al.  Pupil size varies with word listening and response selection difficulty in older adults with hearing loss. , 2013, Psychophysiology.

[22]  M. Just,et al.  From the SelectedWorks of Marcel Adam Just 1992 A capacity theory of comprehension : Individual differences in working memory , 2017 .

[23]  B. Hornsby The Effects of Hearing Aid Use on Listening Effort and Mental Fatigue Associated With Sustained Speech Processing Demands , 2013, Ear and hearing.

[24]  J. Towse,et al.  Individual differences in working memory , 2006, Neuroscience.

[25]  A. M. Mimpen,et al.  Improving the reliability of testing the speech reception threshold for sentences. , 1979, Audiology : official organ of the International Society of Audiology.

[26]  M. McAuliffe,et al.  Effect of speaker age on speech recognition and perceived listening effort in older adults with hearing loss. , 2012, Journal of speech, language, and hearing research : JSLHR.

[27]  Gurjit Singh,et al.  Effects of Age on Auditory and Cognitive Processing: Implications for Hearing Aid Fitting and Audiologic Rehabilitation , 2006, Trends in amplification.

[28]  A. Zekveld,et al.  Cognitive processing load across a wide range of listening conditions: insights from pupillometry. , 2014, Psychophysiology.

[29]  A. Zekveld,et al.  Pupil Response as an Indication of Effortful Listening: The Influence of Sentence Intelligibility , 2010, Ear and hearing.

[30]  D. Kahneman,et al.  Attention and Effort , 1973 .

[31]  Tammo Houtgast,et al.  Occupational performance: Comparing normally-hearing and hearing-impaired employees using the Amsterdam Checklist for Hearing and Work , 2006, International journal of audiology.

[32]  Thomas Lunner,et al.  Cognitive function in relation to hearing aid use , 2003, International journal of audiology.

[33]  Greg J Siegle,et al.  Use of concurrent pupil dilation assessment to inform interpretation and analysis of fMRI data , 2003, NeuroImage.

[34]  D Kahneman,et al.  Pupil Diameter and Load on Memory , 1966, Science.

[35]  P. Carpenter,et al.  Individual differences in working memory and reading , 1980 .

[36]  R. Ornstein,et al.  Pupillary Responses During Information Processing Vary with Scholastic Aptitude Test Scores , 2022 .

[37]  Jean-Pierre Gagné,et al.  Older adults expend more listening effort than young adults recognizing audiovisual speech in noise , 2011, International journal of audiology.

[38]  P. Rabbitt,et al.  Channel-Capacity, Intelligibility and Immediate Memory , 1968, The Quarterly journal of experimental psychology.

[39]  E. Granholm,et al.  Pupillary responses on the visual backward masking task reflect general cognitive ability. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[40]  S. Arlinger,et al.  Speech understanding in quiet and noise, with and without hearing aids , 2005, International journal of audiology.

[41]  R. Hetherington The Shellen chart as a test of visual acuity , 1954, Psychologische Forschung.

[42]  Mads Poulsen,et al.  Context Improves Comprehension of Fronted Objects , 2014, Journal of psycholinguistic research.

[43]  F H Bess,et al.  A comparison of the aided performance and benefit provided by a linear and a two-channel wide dynamic range compression hearing aid. , 1999, Journal of speech, language, and hearing research : JSLHR.

[44]  J M Festen,et al.  Assessing aspects of auditory handicap by means of pupil dilatation. , 1997, Audiology : official organ of the International Society of Audiology.

[45]  Hintat Cheung,et al.  Competing complexity metrics and adults' production of complex sentences , 1992, Applied Psycholinguistics.

[46]  Birger Kollmeier,et al.  Development and evaluation of a linguistically and audiologically controlled sentence intelligibility test. , 2013, The Journal of the Acoustical Society of America.

[47]  B Hagerman,et al.  Clinical measurements of speech reception threshold in noise. , 1984, Scandinavian audiology.

[48]  T. Kobayashi,et al.  Hearing Impairment , 2004, Springer Japan.

[49]  S Arlinger,et al.  Verbal information-processing capabilities and cochlear implants: implications for preoperative predictors of speech understanding. , 1996, Journal of deaf studies and deaf education.

[50]  Isabell Wartenburger,et al.  Resource allocation and fluid intelligence: insights from pupillometry. , 2010, Psychophysiology.

[51]  Jean-Pierre Gagné,et al.  Older adults expend more listening effort than young adults recognizing speech in noise. , 2011, Journal of speech, language, and hearing research : JSLHR.

[52]  Thomas Lunner,et al.  Danish reading span data from 283 hearing-aid users, including a sub-group analysis of their relationship to speech-in-noise performance , 2016, International journal of audiology.

[53]  Prithima R Mosaly,et al.  Quantifying the impact of cross coverage on physician's workload and performance in radiation oncology. , 2013, Practical radiation oncology.

[54]  Birger Kollmeier,et al.  How Hearing Impairment Affects Sentence Comprehension: Using Eye Fixations to Investigate the Duration of Speech Processing , 2015, Trends in hearing.

[55]  J. Rönnberg Cognition in the hearing impaired and deaf as a bridge between signal and dialogue: a framework and a model , 2003, International journal of audiology.

[56]  Thomas Lunner,et al.  Assessing listening effort by measuring short-term memory storage and processing of speech in noise , 2014 .

[57]  J. Brehm,et al.  The intensity of motivation. , 1989, Annual review of psychology.

[58]  A. Wingfield,et al.  Pupillometry as a measure of cognitive effort in younger and older adults. , 2010, Psychophysiology.

[59]  T. Lunner,et al.  Working memory capacity may influence perceived effort during aided speech recognition in noise. , 2012, Journal of the American Academy of Audiology.

[60]  J. Beatty,et al.  Pupillary responses during information processing vary with Scholastic Aptitude Test scores. , 1979, Science.

[61]  A. Wingfield,et al.  Hearing Loss in Older Adulthood , 2005 .

[62]  Line Burholt Kristensen,et al.  Semantic role assignment in Danish children and adults , 2014 .

[63]  R. Plomp A signal-to-noise ratio model for the speech-reception threshold of the hearing impaired. , 1986, Journal of speech and hearing research.