Influence of speaking style adaptations and semantic context on the time course of word recognition in quiet and in noise

Abstract This study examines the effects of different listener-oriented speaking styles and semantic contexts on online spoken word recognition using eyetracking. In Experiment 1, different groups of listeners participated in a word-identification-in-noise and in a pleasantness-rating task. Listeners heard sentences with high- versus low-predictability semantic contexts produced in infant-directed speech, Clear Speech, and Conversational Speech. Experiment 2 (in silence) and 3 (in noise) investigated the time course of visual fixations to target objects when participants were listening to different speaking styles and contexts. Results from all experiments show that relative to conversational speech, both infant-directed speech and Clear Speech improved word recognition for high-predictability sentences, in quiet as well as in noise. This indicates that established advantages of infant-directed speech for young listeners cannot be attributed only to affect; the acoustic enhancements in infant-directed speech benefit adult speech processing as well. Furthermore, in silence (Experiment 2) lexical access was facilitated by contextual cues even in conversational speech; but in noise (Experiment 3) listeners reliably focused the target only when a combination of contextual cues and listener-adapted acoustic–phonetic cues were available. These findings suggest that both semantic cues and listener-oriented acoustic enhancements are needed to facilitate word recognition, especially in adverse listening conditions.

[1]  Reiko Mazuka,et al.  Utterances in infant-directed speech are shorter, not slower , 2016, Cognition.

[2]  Elizabeth K. Johnson,et al.  Toddlers' Language-Mediated Visual Search: They Need not have the Words for It , 2010, Quarterly journal of experimental psychology.

[3]  F Grosjean,et al.  Spoken word recognition processes and the gating paradigm , 1980, Perception & psychophysics.

[4]  Tessa Bent,et al.  The clear speech effect for non-native listeners. , 2002, The Journal of the Acoustical Society of America.

[5]  Arthur Wingfield,et al.  Hearing Loss and Perceptual Effort: Downstream Effects on Older Adults’ Memory for Speech , 2005, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[6]  Rajka Smiljanic,et al.  Effects of intelligibility on within- and cross-modal sentence recognition memory for native and non-native listeners. , 2018, The Journal of the Acoustical Society of America.

[7]  Kathy Hirsh-Pasek,et al.  (Baby)Talk to Me , 2015 .

[8]  G. Altmann Language can mediate eye movement control within 100 milliseconds, regardless of whether there is anything to move the eyes to , 2011, Acta psychologica.

[9]  P. Kuhl,et al.  Maternal speech to infants in a tonal language: Support for universal prosodic features in motherese. , 1988 .

[10]  A. Fernald The perceptual, affective, and linguistic salience of mothers' speech to infants , 1984 .

[11]  Laurence White,et al.  Integration of multiple speech segmentation cues: a hierarchical framework. , 2005, Journal of experimental psychology. General.

[12]  Roger M. Cooper,et al.  The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. , 1974 .

[13]  D. Norris Shortlist: a connectionist model of continuous speech recognition , 1994, Cognition.

[14]  A. Fernald,et al.  Intonation and communicative intent in mothers' speech to infants: is the melody the message? , 1989, Child development.

[15]  Alexander L. Francis,et al.  Effects of intelligibility on working memory demand for speech perception , 2009, Attention, perception & psychophysics.

[16]  R. Aslin,et al.  Preference for infant-directed speech in the first month after birth. , 1990, Child development.

[17]  Peter F. Assmann,et al.  The Perception of Speech Under Adverse Conditions , 2004 .

[18]  M. Daneman,et al.  How young and old adults listen to and remember speech in noise. , 1995, The Journal of the Acoustical Society of America.

[19]  Kris Tjaden,et al.  Acoustics of clear speech: effect of instruction. , 2012, Journal of speech, language, and hearing research : JSLHR.

[20]  Ann R Bradlow,et al.  Temporal organization of English clear and conversational speech. , 2008, The Journal of the Acoustical Society of America.

[21]  Jeffrey Heinz,et al.  Acoustic Characteristics of Infant- directed Speech as a Function of Prosodic Typology , 2016 .

[22]  Rebecca Scarborough,et al.  An acoustic study of real and imagined foreigner‐directed speech , 2007 .

[23]  Katherine Demuth,et al.  Effects of the acoustic properties of infant-directed speech on infant word recognition. , 2010, The Journal of the Acoustical Society of America.

[24]  Matthew P. Aylett,et al.  Taking the hit: Leaving some lexical competition to be resolved post-lexically , 2001 .

[25]  D. Bates,et al.  fitting linear mixed effects models using lme 4 arxiv , 2014 .

[26]  Rajka Smiljanic,et al.  Acoustic and semantic enhancements for children with cochlear implants. , 2013, Journal of speech, language, and hearing research : JSLHR.

[27]  A Boothroyd,et al.  Context effects in phoneme and word recognition by young children and older adults. , 1990, The Journal of the Acoustical Society of America.

[28]  L. Braida,et al.  Speaking clearly for the hard of hearing IV: Further studies of the role of speaking rate. , 1996, Journal of speech and hearing research.

[29]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[30]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[31]  Alexander L. Francis,et al.  Improved segregation of simultaneous talkers differentially affects perceptual and cognitive capacity demands for recognizing speech in competing speech , 2010, Attention, perception & psychophysics.

[32]  T. Lunner,et al.  Cognition counts: A working memory system for ease of language understanding (ELU) , 2008, International journal of audiology.

[33]  A. Friederici,et al.  Lexical integration: Sequential effects of syntactic and semantic information , 1999, Memory & cognition.

[34]  Anne Fernald,et al.  Prosody and focus in speech to infants and adults , 1991 .

[35]  D. Pisoni,et al.  Recognizing Spoken Words: The Neighborhood Activation Model , 1998, Ear and hearing.

[36]  William D. Marslen-Wilson,et al.  Function and process in spoken word recognition: A tutorial review , 1984 .

[37]  P. Kuhl,et al.  Cross-language analysis of phonetic units in language addressed to infants. , 1997, Science.

[38]  K. Demuth,et al.  Phonetic enhancement of Mandarin vowels and tones: Infant-directed speech and Lombard speech. , 2017, The Journal of the Acoustical Society of America.

[39]  Ann R Bradlow,et al.  Semantic and phonetic enhancements for speech-in-noise recognition by native and non-native listeners. , 2007, The Journal of the Acoustical Society of America.

[40]  M. Goldstein,et al.  Habituation, sensitization, and infants' responses to motherese speech. , 1995, Developmental psychobiology.

[41]  Sheng Liu,et al.  Temporal properties in clear speech perception. , 2006, The Journal of the Acoustical Society of America.

[42]  Thomas L. Griffiths,et al.  Infant directed speech is consistent with teaching , 2016, Psychological review.

[43]  Marius Catalin Iordan,et al.  Mothers Consistently Alter Their Unique Vocal Fingerprints When Communicating with Infants , 2017, Current Biology.

[44]  K. Engen Clear speech and lexical competition in younger and older adult listeners , 2017 .

[45]  Elizabeth K. Johnson,et al.  Input-driven differences in toddlers’ perception of a disappearing phonological contrast , 2016 .

[46]  D. Kewley-Port,et al.  Vowel intelligibility in clear and conversational speech , 1999 .

[47]  Reiko Mazuka,et al.  The multidimensional nature of hyperspeech: Evidence from Japanese vowel devoicing , 2014, Cognition.

[48]  M K Tanenhaus,et al.  Eye Movements and Lexical Access in Spoken-Language Comprehension: Evaluating a Linking Hypothesis between Fixations and Linguistic Processing , 2000, Journal of psycholinguistic research.

[49]  Titia Benders,et al.  Mommy is only happy! Dutch mothers' realisation of speech sounds in infant-directed speech expresses emotion, not didactic intent. , 2013, Infant behavior & development.

[50]  Ann R Bradlow,et al.  Production and perception of clear speech in Croatian and English. , 2004, The Journal of the Acoustical Society of America.

[51]  D. Swingley,et al.  Contributions of infant word learning to language development , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[52]  Valerie Hazan,et al.  Acoustic-phonetic correlates of talker intelligibility for adults and children. , 2004, The Journal of the Acoustical Society of America.

[53]  Katharine Graf Estes,et al.  Infant-directed prosody helps infants map sounds to meanings. , 2013, Infancy : the official journal of the International Society on Infant Studies.

[54]  D. Kewley-Port,et al.  Vowel intelligibility in clear and conversational speech for normal-hearing and hearing-impaired listeners. , 2002, The Journal of the Acoustical Society of America.

[55]  R. Aslin,et al.  Spoken word recognition and lexical representation in very young children , 2000, Cognition.

[56]  K. Tjaden,et al.  Intelligibility of clear speech: effect of instruction. , 2013, Journal of speech, language, and hearing research : JSLHR.

[57]  Anne Cutler,et al.  A multimodal corpus of speech to infant and adult listeners. , 2013, The Journal of the Acoustical Society of America.

[58]  Richard N Aslin,et al.  Tracking the time course of phonetic cue integration during spoken word recognition , 2008, Psychonomic bulletin & review.

[59]  K. Tjaden,et al.  Hybridizing conversational and clear speech to investigate the source of increased intelligibility in speakers with Parkinson's disease. , 2014, Journal of speech, language, and hearing research : JSLHR.

[60]  L L Elliott,et al.  Development of a test of speech intelligibility in noise using sentence materials with controlled word predictability. , 1977, The Journal of the Acoustical Society of America.

[61]  L. T. DeCarlo On the meaning and use of kurtosis. , 1997 .

[62]  Bruce A Schneider,et al.  Children's use of semantic cues in degraded listening environments. , 2002, The Journal of the Acoustical Society of America.

[63]  Odette Scharenborg,et al.  The Effect of Background Noise on the Activation of Phonological and Semantic Information During Spoken-Word Recognition , 2016, INTERSPEECH.

[64]  P. Rabbitt,et al.  Channel-Capacity, Intelligibility and Immediate Memory , 1968, The Quarterly journal of experimental psychology.

[65]  Bharath Chandrasekaran,et al.  Effects of Speech Clarity on Recognition Memory for Spoken Sentences , 2012, PloS one.

[66]  Patricia K. Kuhl,et al.  Infant-directed speech in English and Spanish: Assessments of monolingual and bilingual caregiver VOT , 2017, J. Phonetics.

[67]  G. A. Miller,et al.  The intelligibility of speech as a function of the context of the test materials. , 1951, Journal of experimental psychology.

[68]  N. Mani,et al.  Listen up! Developmental differences in the impact of IDS on speech segmentation , 2017, Cognition.

[69]  Boaz M Ben-David,et al.  Effects of aging and noise on real-time spoken word recognition: evidence from eye movements. , 2011, Journal of speech, language, and hearing research : JSLHR.

[70]  Francisco Lacerda,et al.  Voice Onset Time in Speech to Infants and Adults , 1999, Phonetica.

[71]  J. Werker,et al.  Infant preference for both male and female infant-directed talk: a developmental study of attentional and affective responsiveness. , 1989, Canadian journal of psychology.

[72]  A. Jongman,et al.  Acoustic characteristics of clearly spoken English fricatives. , 2009, The Journal of the Acoustical Society of America.

[73]  W. Marslen-Wilson,et al.  Representation and competition in the perception of spoken words , 2002, Cognitive Psychology.

[74]  Rajka Smiljanic,et al.  Acoustics of Clear and Noise-Adapted Speech in Children, Young, and Older Adults. , 2017, Journal of speech, language, and hearing research : JSLHR.

[75]  Patricia K. Kuhl,et al.  Fast Track , 2020, Electric and Hybrid Vehicle Technology International.

[76]  Laurel Fais,et al.  Infant-directed speech supports phonetic category learning in English and Japanese , 2007, Cognition.

[77]  P. Brockhoff,et al.  Tests in Linear Mixed Effects Models , 2015 .

[78]  K. Scherer,et al.  Vocal expression of emotion. , 2003 .

[79]  Daniel Swingley,et al.  Prosodic exaggeration within infant-directed speech: Consequences for vowel learnability. , 2017, The Journal of the Acoustical Society of America.

[80]  John P. Pinto,et al.  Rapid Gains in Speed of Verbal Processing by Infants in the 2nd Year , 1998 .

[81]  Ann R. Bradlow,et al.  Speaking and Hearing Clearly: Talker and Listener Factors in Speaking Style Changes , 2009, Lang. Linguistics Compass.

[82]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[83]  S. Spitzer,et al.  Syllabic strength and lexical boundary decisions in the perception of hypokinetic dysarthric speech. , 1998, The Journal of the Acoustical Society of America.

[84]  Richard Wright,et al.  A new method for eliciting three speaking styles in the laboratory , 2008, Speech Commun..

[85]  N I Durlach,et al.  Speaking clearly for the hard of hearing I: Intelligibility differences between clear and conversational speech. , 1985, Journal of speech and hearing research.

[86]  Yannis Stylianou,et al.  Approaching speech intelligibility enhancement with inspiration from Lombard and Clear speaking styles , 2014, Comput. Speech Lang..

[87]  Kjellrun T. Englund Voice onset time in infant directed speech over the first six months , 2005 .

[88]  M. Cooke,et al.  Recognizing speech under a processing load: Dissociating energetic from informational factors , 2009, Cognitive Psychology.

[89]  Elizabeth K. Johnson,et al.  What infant-directed speech tells us about the development of compensation for assimilation , 2018, J. Phonetics.

[90]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[91]  Matthew H. Davis,et al.  Perceptual and response components in repetition priming of spoken words and pseudowords , 2011, Quarterly journal of experimental psychology.

[92]  Kathy Hirsh-Pasek,et al.  A moment of silence: How the prosodic cues in motherese might assist language learning , 1986 .

[93]  Monja A. Knoll,et al.  Are actresses better simulators than female students? The effects of simulation on prosodic modifications of infant- and foreigner-directed speech , 2009, Speech Commun..

[94]  V. Hazan,et al.  Acoustic-phonetic characteristics of speech produced with communicative intent to counter adverse listening conditions. , 2011, The Journal of the Acoustical Society of America.

[95]  W. Marslen-Wilson,et al.  Accessing Spoken Words: The Importance of Word Onsets , 1989 .

[96]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[97]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[98]  James M. McQueen,et al.  Eight questions about spoken-word recognition , 2007 .

[99]  K. Hirsh-Pasek,et al.  The eyes have it: lexical and syntactic comprehension in a new paradigm , 1987, Journal of Child Language.

[100]  N I Durlach,et al.  Speaking clearly for the hard of hearing. III: An attempt to determine the contribution of speaking rate to differences in intelligibility between clear and conversational speech. , 1989, Journal of speech and hearing research.

[101]  Jean C. Krause,et al.  Properties of naturally produced clear speech at normal rates and implications for intelligibility enhancement , 2001 .

[102]  B. McMurray,et al.  Infant directed speech and the development of speech perception: Enhancing development or an unintended consequence? , 2013, Cognition.

[103]  Susanne Brouwer,et al.  The Temporal Dynamics of Spoken Word Recognition in Adverse Listening Conditions , 2016, Journal of psycholinguistic research.

[104]  Shae D. Morgan,et al.  Judgments of Emotion in Clear and Conversational Speech by Young Adults With Normal Hearing and Older Adults With Hearing Impairment. , 2017, Journal of speech, language, and hearing research : JSLHR.

[105]  David B. Pisoni,et al.  Intelligibility of normal speech I: Global and fine-grained acoustic-phonetic talker characteristics , 1996, Speech Commun..

[106]  Matthew H. Davis,et al.  Speech recognition in adverse conditions: A review , 2012 .

[107]  P. Kuhl Is speech learning 'gated' by the social brain? , 2007, Developmental science.

[108]  D. Dahan The Time Course of Interpretation in Speech Comprehension , 2010 .

[109]  Alejandrina Cristià,et al.  Phonetic enhancement of sibilants in infant-directed speech. , 2010, The Journal of the Acoustical Society of America.

[110]  Susanne Brouwer,et al.  Discourse context and the recognition of reduced and canonical spoken words , 2012, Applied Psycholinguistics.

[111]  P. Jusczyk,et al.  A moment of silence: How the prosodic cues in motherese might assist language learning , 1986 .

[112]  A. Seidl,et al.  The hyperarticulation hypothesis of infant-directed speech* , 2013, Journal of Child Language.

[113]  Paul D. Allopenna,et al.  Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models , 1998 .

[114]  M. Soderstrom,et al.  Beyond babytalk: Re-evaluating the nature and content of speech input to preverbal infants , 2007 .

[115]  Alejandrina Cristià,et al.  Input to Language: The Phonetics and Perception of Infant-Directed Speech , 2013, Lang. Linguistics Compass.

[116]  J. C. Krause,et al.  Acoustic properties of naturally produced clear speech at normal speaking rates. , 1996, The Journal of the Acoustical Society of America.

[117]  R. Shillcock,et al.  The recognition of words after their acoustic offsets in spontaneous speech: Effects of subsequent context , 1988, Perception & psychophysics.

[118]  Nina Kraus,et al.  Speaking clearly for children with learning disabilities: sentence perception in noise. , 2003, Journal of speech, language, and hearing research : JSLHR.

[119]  M. Picheny,et al.  Speaking clearly for the hard of hearing. II: Acoustic characteristics of clear and conversational speech. , 1986, Journal of speech and hearing research.

[120]  Pauline Welby,et al.  The role of early fundamental frequency rises and elbows in French word segmentation , 2007, Speech Commun..

[121]  R. Aslin,et al.  Developmental differences in infant attention to the spectral properties of infant-directed speech. , 1994, Child development.

[122]  Rachael C Gilbert,et al.  Recognition memory in noise for speech of varying intelligibility. , 2012, The Journal of the Acoustical Society of America.

[123]  A. Fernald,et al.  Prosody and focus in speech to infants and adults , 1991 .