Gradient acoustic information induces long-lasting referential uncertainty in short discourses

ABSTRACT Three experiments examined the influence of gradient acoustic information on referential interpretation during spoken language processing and how this influence persists over time. Acoustic continua varying between the pronouns “he” and “she” were created and validated in two offline experiments. A third experiment examined whether these acoustic differences influence online pronoun interpretation, and whether this influence persists across words in a discourse. Measures of eye gaze showed immediate sensitivity to graded acoustic information. Moreover, acoustically induced uncertainty persisted across a five-word delay: When listeners encountered a word that disambiguated the referent of the pronoun differently than it had originally been interpreted, the amount of time they took to recover from an initial misinterpretation was directly related to distance along the acoustic continuum between the pronoun and the endpoint corresponding to the correct referent. These findings show that fine-grained acoustic detail induces referential uncertainty that is maintained over extended periods of time.

[1]  Edward Gibson,et al.  The Processing and Acquisition of Reference , 2011 .

[2]  Mante S. Nieuwland,et al.  Individual differences and contextual bias in pronoun resolution: Evidence from ERPs , 2006, Brain Research.

[3]  M. Pitt,et al.  Lexical activation (and other factors) can mediate compensation for coarticulation , 2003 .

[4]  Paul D. Allopenna,et al.  Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models , 1998 .

[5]  R. Shillcock,et al.  The recognition of words after their acoustic offsets in spontaneous speech: Effects of subsequent context , 1988, Perception & psychophysics.

[6]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[7]  Scott Weinstein,et al.  Centering: A Framework for Modeling the Local Coherence of Discourse , 1995, CL.

[8]  J. Trueswell,et al.  Interpreting pronouns and demonstratives in Finnish: Evidence for a form-specific approach to reference resolution , 2008 .

[9]  H. Rohde,et al.  Integration of pragmatic and phonetic cues in spoken word recognition. , 2012, Journal of experimental psychology. Learning, memory, and cognition.

[10]  Mante S. Nieuwland,et al.  Establishing reference in language comprehension: An electrophysiological perspective , 2007, Brain Research.

[11]  G. Dehaene-Lambertz Electrophysiological correlates of categorical phoneme perception in adults , 1997, Neuroreport.

[12]  Panagiotis G. Ipeirotis Analyzing the Amazon Mechanical Turk marketplace , 2010, XRDS.

[13]  Joseph C. Toscano,et al.  Continuous Perception and Graded Categorization , 2010, Psychological science.

[14]  Jennifer E. Arnold,et al.  The effect of additional characters on choice of referring expression: Everyone counts. , 2007, Journal of memory and language.

[15]  Michael J. Spivey,et al.  Continuous attraction toward phonological competitors. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Rachel S. Sussman,et al.  Structural and semantic constraints on the resolution of pronouns and reflexives , 2009, Cognition.

[17]  B. C. Griffith,et al.  The discrimination of speech sounds within and across phoneme boundaries. , 1957, Journal of experimental psychology.

[18]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[19]  Hannah Rohde,et al.  A probabilistic reconciliation of coherence-driven and centering-driven theories of pronoun interpretation , 2013 .

[20]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[21]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[22]  D. Norris,et al.  No lexical–prelexical feedback during speech perception or: Is it time to stop playing those Christmas tapes? , 2009 .

[23]  Julie C. Sedivy,et al.  The effect of speaker-specific information on pragmatic inferences , 2011 .

[24]  Dawn G. Blasko,et al.  Effects of subsequent sentence context in auditory word recognition: Temporal and linguistic constrainst , 1991 .

[25]  K. Rayner,et al.  Eye movement evidence that readers maintain and act on uncertainty about past linguistic input , 2009, Proceedings of the National Academy of Sciences.

[26]  K. Nakayama,et al.  Is the Web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments , 2012, Psychonomic Bulletin & Review.

[27]  M. Tanenhaus,et al.  Continuous mapping from sound to meaning in spoken-language comprehension: immediate effects of verb-based thematic constraints. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[28]  Hannah Rohde,et al.  Grammatical and information-structural influences on pronoun production , 2014 .

[29]  E. Chang,et al.  Categorical Speech Representation in Human Superior Temporal Gyrus , 2010, Nature Neuroscience.

[30]  Michael D. Buhrmester,et al.  Amazon's Mechanical Turk , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[31]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[32]  D. Swinney Lexical access during sentence comprehension: (Re)consideration of context effects , 1979 .

[33]  Sarah Brown-Schmidt,et al.  Journal of Experimental Psychology : General Perspective-Taking in Comprehension , Production , and Memory : An Individual Differences Approach , 2015 .

[34]  Jeanette K. Gundel,et al.  Cognitive Status and the Form of Referring Expressions in Discourse , 1993 .

[35]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[36]  S. Goldinger Words and voices: episodic traces in spoken word identification and recognition memory. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[37]  M. Tanenhaus,et al.  Gradient effects of within-category phonetic variation on lexical access , 2002, Cognition.

[38]  Sarah Brown-Schmidt,et al.  The rapid use of gender information: evidence of the time course of pronoun resolution from eyetracking , 2000, Cognition.

[39]  S. Blumstein,et al.  The effect of subphonetic differences on lexical access , 1994, Cognition.

[40]  Sarah Brown-Schmidt,et al.  Hippocampal contributions to language: evidence of referential processing deficits in amnesia. , 2013, Journal of experimental psychology. General.

[41]  Lori L. Holt,et al.  Are there interactive processes in speech perception? , 2006, Trends in Cognitive Sciences.

[42]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[43]  Barbara J. Grosz,et al.  Pronouns, Names, and the Centering of Attention in Discourse , 1993, Cogn. Sci..

[44]  D. Dahan The Time Course of Interpretation in Speech Comprehension , 2010 .

[45]  W. Ganong Phonetic categorization in auditory word perception. , 1980, Journal of experimental psychology. Human perception and performance.

[46]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[47]  M. Tanenhaus,et al.  Within-category VOT affects recovery from "lexical" garden paths: Evidence against phoneme-level inhibition. , 2009, Journal of memory and language.

[48]  Jennifer E. Arnold,et al.  If you say thee uh you are describing something hard: the on-line attribution of disfluency during reference comprehension. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[49]  R. Ratcliff,et al.  Unheralded Pronouns and Management by Common Ground , 1994 .

[50]  D. Dahan,et al.  The temporal dynamics of ambiguity resolution: Evidence from spoken-word recognition. , 2007, Journal of memory and language.