An Eye-Tracking Paradigm for Analyzing the Processing Time of Sentences with Different Linguistic Complexities

An eye-tracking paradigm was developed for use in audiology in order to enable online analysis of the speech comprehension process. This paradigm should be useful in assessing impediments in speech processing. In this paradigm, two scenes, a target picture and a competitor picture, were presented simultaneously with an aurally presented sentence that corresponded to the target picture. At the same time, eye fixations were recorded using an eye-tracking device. The effect of linguistic complexity on language processing time was assessed from eye fixation information by systematically varying linguistic complexity. This was achieved with a sentence corpus containing seven German sentence structures. A novel data analysis method computed the average tendency to fixate the target picture as a function of time during sentence processing. This allowed identification of the point in time at which the participant understood the sentence, referred to as the decision moment. Systematic differences in processing time were observed as a function of linguistic complexity. These differences in processing time may be used to assess the efficiency of cognitive processes involved in resolving linguistic complexity. Thus, the proposed method enables a temporal analysis of the speech comprehension process and has potential applications in speech audiology and psychoacoustics.

[1]  A. M. Mimpen,et al.  Improving the reliability of testing the speech reception threshold for sentences. , 1979, Audiology : official organ of the International Society of Audiology.

[2]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[3]  Martin Hansen,et al.  Measurement and prediction of the acceptable noise level for single-microphone noise reduction algorithms , 2012, International journal of audiology.

[4]  Timothy M. Gann,et al.  Anticipatory baseline effects and information integration in visual world studies. , 2011, Acta psychologica.

[5]  J. Cerella,et al.  The rise and fall in information-processing rates over the life span. , 1994, Acta psychologica.

[6]  Markus Bader,et al.  Subject-Object Ambiguities in German Embedded Clauses: An Across-the-Board Comparison , 1999 .

[7]  A. Boothroyd,et al.  Mathematical treatment of context effects in phoneme and word recognition. , 1988, The Journal of the Acoustical Society of America.

[8]  S. Soli,et al.  Development of the Hearing in Noise Test for the measurement of speech reception thresholds in quiet and in noise. , 1994, The Journal of the Acoustical Society of America.

[9]  L L Elliott,et al.  Development of a test of speech intelligibility in noise using sentence materials with controlled word predictability. , 1977, The Journal of the Acoustical Society of America.

[10]  Falk Huettig,et al.  The tug of war between phonological, semantic and shape information in language-mediated visual search , 2007 .

[11]  T. Zandt.  Analysis of Response Time Distributions , 2002 .

[12]  Randall Hendrick,et al.  Memory-Load Interference in Syntactic Processing , 2002, Psychological science.

[13]  J. Trueswell,et al.  The developing constraints on parsing decisions: The role of lexical-biases and referential scenes in child and adult sentence processing , 2004, Cognitive Psychology.

[14]  R. Bentler.  Effectiveness of directional microphones and noise reduction schemes in hearing aids: a systematic review of the evidence. , 2005, Journal of the American Academy of Audiology.

[15]  A. Wingfield,et al.  Cognitive supports and cognitive constraints on comprehension of spoken language. , 2007, Journal of the American Academy of Audiology.

[16]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[17]  Gisbert Fanselow,et al.  Contextual Licensing of Marked OVS Word Order in German  , 2011, Linguistische Berichte (LB).

[18]  Arthur Wingfield,et al.  Response latencies in auditory sentence comprehension: effects of linguistic versus perceptual challenge. , 2010, Psychology and aging.

[19]  Anita Steube,et al.  The discourse potential of underspecified structures , 2008 .

[20]  Josef Bayer,et al.  Case and Linking in Language Comprehension: Evidence from German , 2006 .

[21]  Birger Kollmeier,et al.  How does linguistic complexity influence intelligibility in a German audiometric sentence intelligibility test? , 2011, International journal of audiology.

[22]  Joseph C. Toscano,et al.  Cue-integration and context effects in speech: Evidence against speaking-rate normalization , 2012, Attention, perception & psychophysics.

[23]  E. Gibson.  The dependency locality theory: A distance-based theory of linguistic complexity. , 2000 .

[24]  Esther Ruigendijk,et al.  The Effects of Syntactic Complexity on Processing Sentences in Noise , 2013, Journal of psycholinguistic research.

[25]  G. Altmann,et al.  The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing , 2007 .

[26]  Julie C. Sedivy,et al.  Eye movements as a window into real-time spoken language comprehension in natural contexts , 1995, Journal of psycholinguistic research.

[27]  G. Studebaker,et al.  Supplementary formulas and tables for calculating and interconverting speech recognition scores in transformed arcsine units , 2004, International journal of audiology.

[28]  Y. Miyashita,et al.  Image, language, brain , 2000 .

[29]  B Hagerman,et al.  Sentences for testing speech intelligibility in noise. , 1982, Scandinavian audiology.

[30]  Matthew W. Crocker,et al.  The influence of recent scene events on spoken comprehension: Evidence from eye movements , 2007 .

[31]  Matthew W. Crocker,et al.  The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye-movements in depicted events , 2005, Cognition.

[32]  Boaz M Ben-David,et al.  Effects of aging and noise on real-time spoken word recognition: evidence from eye movements. , 2011, Journal of speech, language, and hearing research : JSLHR.

[33]  Pia Knoeferle,et al.  Comparing the time-course of processing initially ambiguous and unambiguous German SVO/OVS sentences in depicted events , 2007 .

[34]  Thomas Lenarz,et al.  Indication criteria for cochlear implants and hearing aids: impact of audiological and non-audiological findings , 2012, Audiology research.

[35]  Sridhar Kalluri,et al.  Objective measures of listening effort: effects of background noise and noise reduction. , 2009, Journal of speech, language, and hearing research : JSLHR.

[36]  G. Altmann,et al.  Incremental interpretation at verbs: restricting the domain of subsequent reference , 1999, Cognition.

[37]  Mark Marzinzik,et al.  Noise Reduction Schemes for Digital Hearing Aids and Their Use for the Hearing Impaired , 2001 .

[38]  M. Tanenhaus,et al.  Circumscribing Referential Domains during Real-Time Language Comprehension , 2002 .

[39]  Anita Steube.  Studies on the Acceptability of Object Movement to Spec,CP , 2008 .

[40]  Robin L. Hill,et al.  Eye movements : a window on mind and brain , 2007 .

[41]  A. Wingfield,et al.  Pupillometry as a measure of cognitive effort in younger and older adults. , 2010, Psychophysiology.

[42]  Birger Kollmeier,et al.  Development and evaluation of a linguistically and audiologically controlled sentence intelligibility test. , 2013, The Journal of the Acoustical Society of America.

[43]  B Kollmeier,et al.  Development and evaluation of a German sentence test for objective and subjective speech intelligibility assessment. , 1997, The Journal of the Acoustical Society of America.

[44]  A. Meyer,et al.  Using the visual world paradigm to study language processing: a review and critical evaluation. , 2011, Acta psychologica.

[45]  Jennifer E. Arnold,et al.  Disfluencies Signal Theee, Um, New Information , 2003, Journal of psycholinguistic research.

[46]  J. Trueswell,et al.  Interpreting pronouns and demonstratives in Finnish: Evidence for a form-specific approach to reference resolution , 2008 .

[47]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[48]  Paul D. Allopenna,et al.  Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models , 1998 .

[49]  Anna Warzybok,et al.  Polish sentence matrix test for speech intelligibility measurement in noise , 2010, International journal of audiology.

[50]  Lars Konieczny,et al.  German Sentence Processing , 1999 .

[51]  H. Pashler.  STEVENS' HANDBOOK OF EXPERIMENTAL PSYCHOLOGY , 2002 .

[52]  Wouter A Dreschler,et al.  Perceptual Effects of Noise Reduction With Respect to Personal Preference, Speech Intelligibility, and Listening Effort , 2013, Ear and hearing.

[53]  Michael D. Dodd,et al.  Growing Older Does Not Always Mean Moving Slower: Examining Aging and the Saccadic Motor System , 2006, Journal of motor behavior.

[54]  Arthur Wingfield,et al.  Effects of adult aging and hearing loss on comprehension of rapid speech varying in syntactic complexity. , 2006, Journal of the American Academy of Audiology.

[55]  Birger Kollmeier,et al.  Intelligibility of time-compressed speech: the effect of uniform versus non-uniform time-compression algorithms. , 2014, The Journal of the Acoustical Society of America.

[56]  Anna Warzybok,et al.  Speech-in-noise tests for multilingual hearing screening and diagnostics1. , 2013, American journal of audiology.

[57]  Richard N Aslin,et al.  Tracking the time course of phonetic cue integration during spoken word recognition , 2008, Psychonomic bulletin & review.

[58]  C Laroche,et al.  An approach to the development of hearing standards for hearing-critical jobs. , 2003, Noise & health.

[59]  Paul Gorrell,et al.  The Subject-Before-Object Preference in German Clauses , 2000 .

[60]  Matthew W. Crocker,et al.  The Coordinated Interplay of Scene, Utterance, and World Knowledge: Evidence From Eye Tracking , 2006, Cogn. Sci..

[61]  G. Altmann.  Ambiguity in sentence processing , 1998, Trends in Cognitive Sciences.

[62]  Chantal Laroche,et al.  Functionally-based screening criteria for hearing-critical jobs based on the Hearing in Noise Test , 2008, International journal of audiology.

[63]  Roger M. Cooper,et al.  The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. , 1974 .