"Excuse meeee!!": (Mis)coordination of lexical and paralinguistic prosody in L2 hyperarticulation

Abstract The present study investigates how first language (L1) and second language (L2) speakers coordinate lexical and paralinguistic prosody (lexically determined pitch accent and utterance-level intonation) in hyperarticulated speech. Modifications of F0 and segmental durations in repeated utterances were analyzed jointly using a novel method, Functional Principal Component Analysis (FPCA). By testing Japanese and German participants as L1 and as L2 speakers of the respective languages, the task elicited a situation in which L2 speakers had to handle F0 and segmental durations when these prosodic properties mainly carry a paralinguistic function in their L1, but carry a lexical one in their L2 or vice versa. Results show language-dependent and -independent modifications in hyperarticulated speech in both speaker groups. The language-dependent modifications were negatively transferred to the respective L2. The restriction of Japanese lexical pitch accent prevented Japanese participants from phonologically modifying F0 contours in L1 and L2, and German L2 speakers applied the German pitch accent rules in L2 Japanese utterances. Lexical rules outweighed paralinguistic rules in modifying prosody. At the same time, both German and Japanese speakers hyperarticulated the utterances with utterance-final lengthening, larger pitch range and higher peak. Furthermore, we argue that there may be different dominance orders of prosodic properties in Japanese and German according to which prosodic property was more likely to be modified in hyperarticulated speech. L2 speakers were more prone to change a more dominant prosodic property in their L1 to convey a paralinguistic meaning, i.e., segmental durations for Japanese and F0 for German. The findings are particularly noteworthy as the uttered words were highly frequent words of which L2 learners should have had sufficient L2 input before.

[1]  S. Jun,et al.  Prosodic typology : the phonology of intonation and phrasing , 2014 .

[2]  Martine Grice,et al.  Deutsche Intonation und GToBI , 2002 .

[3]  Keikichi Hirose,et al.  Comparison of acoustic features of word accent in English and Japanese. , 1986 .

[4]  S. Oyama A sensitive period for the acquisition of a nonnative phonological system , 1976 .

[5]  Stefanie Shattuck-Hufnagel,et al.  A prosody tutorial for investigators of auditory sentence processing , 1996, Journal of psycholinguistic research.

[6]  Etsuko Ofuka,et al.  Prosodic cues for rated politeness in Japanese speech , 2000, Speech Commun..

[7]  Caroline Féry,et al.  How focus and givenness shape prosody , 2009 .

[8]  Martin Cooke,et al.  DIAPIX-FL: a symmetric corpus of problem-solving dialogues in first and second languages , 2014, INTERSPEECH.

[9]  H. Pirker,et al.  I SAID TWO TI CKETS HOW TO TALK TO A DEAF WIZARD , 1999 .

[10]  T. Stivers,et al.  A Scalar View of Response Relevance , 2010 .

[11]  K. Pike,et al.  The intonation of American English , 1946 .

[12]  M. Kondo Strategies for Acquiring Japanese Prosody by English Speakers , 2005 .

[13]  A. Botinis,et al.  Intonation , 2001, Speech Commun..

[14]  A. Fernald,et al.  Intonation and communicative intent in mothers' speech to infants: is the melody the message? , 1989, Child development.

[15]  M. Beckman Stress And Non-Stress Accent , 1986 .

[16]  Bettina Braun,et al.  Pitch accent distribution in German infant-directed speech , 2015, INTERSPEECH.

[17]  Kikuo Maekawa,et al.  Prominence Marking in the Japanese Intonation System , 2008 .

[18]  M. Hielscher-Fastabend,et al.  Emotional and Linguistic Perception of Prosody , 2004, Folia Phoniatrica et Logopaedica.

[19]  Rein Ove Sikveland,et al.  How do we Speak to Foreigners? – Phonetic Analyses of Speech Communication between L1 and L2 Speakers of Norwegian , 2009 .

[20]  Mary E. Beckman,et al.  Discourse structure and attentional salience effects on japanese intonation , 2000 .

[21]  Timothy J. Vance,et al.  An introduction to Japanese phonology , 1987 .

[22]  C. R. Deboor,et al.  A practical guide to splines , 1978 .

[23]  Julia Hirschberg,et al.  Prosody and speaker state: paralinguistics, pragmatics, and proficiency , 2007 .

[24]  S. Baumann,et al.  AN INTRODUCTION TO INTONATION – FUNCTIONS AND MODELS , 2006 .

[25]  Anton Batliner,et al.  Consistency in transcription and labelling of German intonation with GToBI , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[26]  Murray J. Munro,et al.  Effects of age of second-language learning on the production of English consonants , 1995, Speech Commun..

[27]  Shinichiro Ishihara,et al.  Japanese focus prosody revisited: Freeing focus from prosodic phrasing , 2011 .

[28]  Astrid Kraehenmann,et al.  Swiss German stops: geminates all over the word , 2001, Phonology.

[29]  A. Paeschke,et al.  F0-CONTOURS IN EMOTIONAL SPEECH , 1999 .

[30]  Hagen Soltau,et al.  Specialized acoustic models for hyperarticulated speech , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[31]  Astrid Paeschke,et al.  Prosodic Characteristics of Emotional Speech: Measurements of Fundamental Frequency Movements , 2000 .

[32]  Sun-Ah Jun,et al.  Acquisition of second language intonation , 2000, INTERSPEECH.

[33]  Marc Schröder,et al.  Voice quality interpolation for emotional text-to-speech synthesis , 2005, INTERSPEECH.

[34]  T. Sekiguchi,et al.  The Use of Lexical Prosody for Lexical Access of the Japanese Language , 1999 .

[35]  J. Flege Age of learning and second language speech. , 1999 .

[36]  K. Bot,et al.  The Role of Intonation in Foreign Accent , 1987 .

[37]  P. Kuhl Human adults and human infants show a “perceptual magnet effect” for the prototypes of speech categories, monkeys do not , 1991, Perception & psychophysics.

[38]  R. Port,et al.  Evidence for mora timing in Japanese. , 1987, The Journal of the Acoustical Society of America.

[39]  P. Kuhl,et al.  Maternal speech to infants in a tonal language: Support for universal prosodic features in motherese. , 1988 .

[40]  A. Fernald,et al.  Expanded Intonation Contours in Mothers' Speech to Newborns. , 1984 .

[41]  Miriam Kienast,et al.  Acoustical analysis of spectral and temporal changes in emotional speech , 2000 .

[42]  T. Arai,et al.  The role of the mora in the timing of spontaneous Japanese speech. , 2001, The Journal of the Acoustical Society of America.

[43]  I. Lehiste The Timing of Utterances and Linguistic Boundaries , 1972 .

[44]  Ulrike Gut Prosody in second language speech production: the role of the native language : Mündliche Produktion in der Fremdsprache , 2003 .

[45]  D. Robert Ladd,et al.  Effects of foot structure on mora duration in Japanese ? , 2003 .

[46]  Aoju Chen,et al.  Universal and language-specific perception of paralinguistic intonational meaning , 2005 .

[47]  D. Stern,et al.  The prosody of maternal speech: infant age and context related changes , 1983, Journal of Child Language.

[48]  Ingo Plag,et al.  Word stress assignment in German, English and Dutch: Quantity-sensitivity and extrametricality revisited , 2014 .

[49]  C. Best A direct realist view of cross-language speech perception , 1995 .

[50]  Roland Göcke,et al.  Auditory-visual infant directed speech in Japanese and English , 2009, AVSP.

[51]  Hagen Soltau,et al.  Phone dependent modeling of hyperarticulated effects# , 2000, INTERSPEECH.

[52]  Jianwu Dang,et al.  Emotional intonation modeling: A cross-language study on Chinese and Japanese , 2013, 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.

[53]  Judit Kormos,et al.  Speech production and second language acquisition , 2006 .

[54]  Marie Frei,et al.  Functional Data Analysis With R And Matlab , 2016 .

[55]  Astrid Paeschke,et al.  A database of German emotional speech , 2005, INTERSPEECH.

[56]  Haruo Kubozono,et al.  Mora and Syllable , 2017 .

[57]  Denis Jouvet,et al.  The IFCASL Corpus of French and German Non-native and Native Read Speech , 2016, LREC.

[58]  Ann Wennerstrom,et al.  Intonational Meaning in English Discourse: A Study of Non-Native Speakers. , 1994 .

[59]  J. Venditti Japanese ToBI Labelling Guidelines , 1997 .

[60]  J. Cohn,et al.  A combination of vocal fo dynamic and summary features discriminates between three pragmatic categories of infant-directed speech. , 1996, Child development.

[61]  Bettina Braun,et al.  On-line interpretation of intonational meaning in L2 , 2011 .

[62]  Y. Asano Stability in perceiving non-native segmental length contrasts , 2014 .

[63]  D. Burnham,et al.  Universality and specificity in infant-directed speech: Pitch modifications as a function of infant age and sex in a tonal and non-tonal language , 2001 .

[64]  Aditi Lahiri,et al.  Duration differences in the articulation and acoustics of Swiss German word-initial geminate and singleton stops. , 2008, The Journal of the Acoustical Society of America.

[65]  Yan Xiao,et al.  Second Language Acquisition: An Introductory Course , 2014 .

[66]  Yuki Asano,et al.  Coordination of lexical and paralinguistic F0 in L2 production , 2015, ICPhS.

[67]  S. Baumann,et al.  TONAL AND ARTICULATORY MARKING OF FOCUS IN GERMAN , 2007 .

[68]  David A. Belsley,et al.  Regression Analysis and its Application: A Data-Oriented Approach.@@@Applied Linear Regression.@@@Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1981 .

[69]  Anne Wichmann,et al.  THE ATTITUDINAL EFFECTS OF PROSODY , AND HOW THEY RELATE TO EMOTION , 2000 .

[70]  Fredrik Karlsson,et al.  Characterising the internal structure of learner intonation and its development over time , 2010 .

[71]  周彬彬,et al.  Interlanguage : forty years later , 2014 .

[72]  Ulrike Gut,et al.  An introduction to intonation – functions and models , 2007 .

[73]  窪薗 晴夫,et al.  The organization of Japanese prosody , 1987 .

[74]  Tsutomu Sugawara,et al.  A Problem of Prosodic Transfer in the Perception of German Learners of Japanese and Japanese Learners of German , 2011, ICPhS.

[75]  C. Best,et al.  Nonnative and second-language speech perception : commonalities and complementarities , 2007 .

[76]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[77]  Michele Gubian,et al.  A Quantitative Investigation of the Prosody of Verum Focus in Italian , 2011, INTERSPEECH.

[78]  Richard Wiese,et al.  The Phonology of German , 1996 .

[79]  A. Cutler,et al.  Pitch accent in spoken-word recognition in Japanese. , 1999, The Journal of the Acoustical Society of America.

[80]  Ian Cunnings,et al.  An overview of mixed-effects statistical models for second language researchers , 2012 .

[81]  C. A. Ferguson Baby Talk in Six Languages , 1964 .

[82]  S E Trehub,et al.  Music and speech processing in the first year of life. , 1993, Advances in child development and behavior.

[83]  J. Trouvain,et al.  Phonological and phonetic influences in non-native intonation , 2007 .

[84]  Michele Gubian,et al.  Cutting down on manual pitch contour annotation using data modelling , 2016 .

[85]  C. Gussenhoven The phonology of tone and intonation , 2004 .

[87]  S. Baumann,et al.  German Intonation in Autosegmental-Metrical Phonology* , 2005 .

[88]  Ulrike Gut,et al.  Non-native prosody : phonetic description and teaching practice , 2007 .

[89]  Hans-Georg Müller,et al.  Functional Data Analysis , 2016 .

[90]  Hagen Soltau,et al.  Compensating hyperarticulation for automatic speech recognition , 2005 .

[91]  Ineke Mennen,et al.  Bi-directional interference in the intonation of Dutch speakers of Greek , 2004, J. Phonetics.

[92]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[93]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[94]  D. Ladd Stylized intonation , 2015 .

[95]  S Oviatt,et al.  Modeling global and focal hyperarticulation during human-computer error resolution. , 1998, The Journal of the Acoustical Society of America.

[96]  Y. Asano Localising foreign accents in speech perception, storage and production , 2016 .

[97]  Sharon L. Oviatt,et al.  Predicting hyperarticulate speech during human-computer error resolution , 1998, Speech Commun..

[98]  Suiping Wang,et al.  The Effect of Intensified Language Exposure on Accommodating Talker Variability. , 2015, Journal of speech, language, and hearing research : JSLHR.

[99]  Shuangzhe Liu,et al.  Regression diagnostics , 2020, Applied Quantitative Analysis for Real Estate.

[100]  Caroline Féry,et al.  German intonational patterns , 1993 .

[101]  Eva Gårding CONTRASTIVE PROSODY: A MODEL AND ITS APPLICATION , 1981 .

[102]  Martine Grice,et al.  the transcription of German intonation 1 GToBI-a phonological system for the transcription of German intonation , 2000 .

[103]  Hiroyuki Nagahara,et al.  Phonological phrasing in Japanese , 1994 .

[104]  Amanda Stent,et al.  Adapting speaking after evidence of misrecognition: Local and global hyperarticulation , 2008, Speech Commun..

[105]  P Taylor,et al.  Analysis and synthesis of intonation using the Tilt model. , 2000, The Journal of the Acoustical Society of America.

[106]  Martine Grice,et al.  The intonation of accessibility , 2006 .

[107]  Lou Boves,et al.  Joint analysis of f0 and speech rate with Functional Data Analysis , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[108]  V. Aubergé,et al.  Recognition of Japanese attitudes in Audio-Visual speech , 2008 .

[109]  M. Papoušek Early ontogeny of vocal communication in parent-infant interactions , 1992 .

[110]  Lou Boves,et al.  Using Functional Data Analysis for investigating multidimensional dynamic phonetic contrasts , 2015, J. Phonetics.

[111]  Daniel Hirst,et al.  Automatic modelling of fundamental frequency using a quadratic sline function , 1993 .

[112]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[113]  Kikuo Maekawa Production and Perception of ‘Paralinguistic’ Information , 2003 .

[114]  C. Best,et al.  Discrimination of non-native consonant contrasts varying in perceptual assimilation to the listener's native phonological system. , 2001, The Journal of the Acoustical Society of America.

[115]  James Emil Flege,et al.  Assessing bilingual dominance , 2002 .

[116]  Willem J. M. Levelt,et al.  Producing spoken language: A blueprint of the speaker , 1999 .

[117]  Michele Gubian,et al.  L1 Prosodic transfer and priming effects: A quantitative study on semi-spontaneous dialogues , 2012 .

[118]  D. Burnham,et al.  Pitch and Communicative Intent in Mother's Speech: Adjustments for Age and Sex in the First Year , 2003 .

[119]  Ann Wennerstrom,et al.  INTONATION AS COHESION IN ACADEMIC DISCOURSE , 1998, Studies in Second Language Acquisition.

[120]  Aoju Chen,et al.  Language-Specificity in the Perception of Paralinguistic Intonational Meaning , 2004, Language and speech.

[121]  Judit Kormos Sentence Production in a Second Language , 2012 .

[122]  Paolo Roseano,et al.  Transcription of Intonation of the Spanish Language , 2010 .

[123]  Bettina Braun,et al.  Phonetics and Phonology of Thematic Contrast in German , 2006, Language and speech.

[124]  P. Kuhl,et al.  Linguistic experience and the "perceptual magnet effect." , 1995 .

[125]  Stefanie Shattuck-Hufnagel,et al.  The original ToBI system and the evolution of the ToBI framework , 2003 .

[126]  K. Kohler Prosody in Speech Interaction Expression of the Speaker and Appeal to the Listener , 2008 .

[127]  Greg Kochanski,et al.  Connecting Intonation Labels to Mathematical Descriptions of Fundamental Frequency , 2007, Language and speech.

[128]  L. Trainor,et al.  The Acoustic Basis of Preferences for Infant-Directed Singing , 1997 .

[129]  Haruo Kubozono,et al.  Focus and Intonation in Japanese: Does Focus Trigger Pitch Reset? * , 2005 .

[130]  Klaus J. Kohler,et al.  Timing and Communicative Functions of Pitch Contours , 2005, Phonetica.

[131]  N. Cowan Attention and Memory: An Integrated Framework , 1995 .

[132]  A. Paeschke Global Trend of Fundamental Frequency in Emotional Speech , 2004 .