Speech melody as articulatorily implemented communicative functions

The understanding of speech melody, i.e., pitch variations related to tone and intonation, can be improved by simultaneously taking into consideration two basic facts: that speech conveys communicative meanings, and that it is produced by human articulators. Communicative meanings, as I will argue, are conveyed through a set of separate functions that are realized by an articulatory system with various biophysical properties. These properties make it unlikely that the melodic functions are encoded directly in terms of invariant surface acoustic forms. Rather, the encoding is likely done through the manipulation of a limited number of articulatorily operable parameters that may be considered as the phonetic primitives. Four such primitives can be recognized for speech melody: local pitch targets, pitch range, articulatory strength and duration. The values of the melodic primitives are specified by a set of encoding schemes, each associated with a particular communicative function. The encoding schemes are distinct from each other in the manner of controlling the melodic primitives, which allows multiple communicative functions to be conveyed in parallel. The communicative functions are ultimately converted to continuous, detailed surface acoustic patterns through an articulatory process of syllable-synchronized sequential target approximation, which takes the melodic primitives specified by the encoding schemes as the control parameters. This view of speech melody is summarized into a comprehensive model of tone and intonation, namely, the parallel encoding and target approximation (PENTA) model.

[1]  James F. Allen,et al.  A Study on Prosody and Discourse Structure in Cooperative Dialogues , 1993 .

[2]  Yi Xu,et al.  THREE LEVELS OF TUNE-TEXT RELATIONS , 2003 .

[3]  Matthew Y. Chen,et al.  Tone Sandhi: Patterns across Chinese Dialects , 2000 .

[4]  Janet B. Pierrehumbert,et al.  Tonal Elements and Their Alignment , 2000 .

[5]  D. Bolinger Around the Edge of Language: Intonation , 1964 .

[6]  San Duanmu,et al.  Syllabic weight and syllabic duration: a correlation between phonology and phonetics , 1994, Phonology.

[7]  Yi Xu,et al.  Production of Weak Elements in Speech – Evidence from F₀ Patterns of Neutral Tone in Standard Chinese , 2006, Phonetica.

[8]  Yi Xu Articulatory Constraints and Tonal alignment , 2002 .

[9]  J. Bernstein,et al.  Fundamental frequency in sentence production , 1984, Proceedings of the IEEE.

[10]  Chilin Shih,et al.  Prosody modeling with soft templates , 2003, Speech Commun..

[11]  J. Kelso Phase transitions and critical behavior in human bimanual coordination. , 1984, The American journal of physiology.

[12]  George N. Clements,et al.  Downstep and high raising: interacting factors in Yoruba tone production , 2003, J. Phonetics.

[13]  Juliette Blevins,et al.  A Tonal Analysis of Lithuanian Nominal Accent. , 1993 .

[14]  I. Titze,et al.  A theoretical study of the effects of various laryngeal configurations on the acoustics of phonation. , 1979, The Journal of the Acoustical Society of America.

[15]  Yi Xu Transmitting Tone and Intonation Simultaneously — The Parallel Encoding and Target Approximation ( PENTA ) Model , 2004 .

[16]  Scott Myers,et al.  Surface underspecification of tone in Chichewa , 1998, Phonology.

[17]  Yi Xu,et al.  Maximum speed of pitch change and how it may relate to speech. , 2002, The Journal of the Acoustical Society of America.

[18]  Bruce Connell,et al.  Tone , Utterance Length and F 0 Scaling , 2004 .

[19]  John J. Ohala,et al.  Duration related phase realignment of Thai tones , 2002, INTERSPEECH.

[20]  J. Ohala Cross-Language Use of Pitch: An Ethological View , 1983, Phonetica.

[21]  Koel Das,et al.  The effects of F0 manipulation on the perceived distance of speech , 2002, INTERSPEECH.

[22]  G. Fairbanks Voice and articulation drillbook , 1960 .

[23]  Kim E. A. Silverman,et al.  The timing of prenuclear high accents in English , 1987 .

[24]  Amalia Arvaniti,et al.  On the place of phrase accents in intonational phonology , 2000, Phonology.

[25]  Stefanie Shattuck-Hufnagel,et al.  A prosody tutorial for investigators of auditory sentence processing , 1996, Journal of psycholinguistic research.

[26]  Philip R. Cohen,et al.  Intentions in Communication. , 1992 .

[27]  D. Ladd,et al.  Constant "segmental anchoring" of F0 movements under changes in speech rate. , 1999, The Journal of the Acoustical Society of America.

[28]  Peter F. MacNeilage,et al.  The Production of Speech , 2011, Springer New York.

[29]  Draga Zec,et al.  The Phonology-syntax connection , 1990 .

[30]  A. Cohen,et al.  Structure and Process in Speech Perception , 1975 .

[31]  A. Wichmann Attitudinal intonation and the inferential process , 2002, Speech Prosody 2002.

[32]  Johan Sundberg,et al.  Maximum speed of pitch changes in singers and untrained subjects , 1979 .

[33]  Yi Xu Contextual tonal variations in Mandarin , 1997 .

[34]  Shinji Maeda,et al.  A characterization of American English intonation , 1976 .

[35]  René Collier,et al.  On the perceptual analysis of intonation , 1990, Speech Commun..

[36]  S. Ohman Word and sentence intonation, a quantitative model , 1967 .

[37]  Yi Xu,et al.  THE PENTA MODEL OF SPEECH MELODY: TRANSMITTING MULTIPLE COMMUNICATIVE FUNCTIONS IN PARALLEL , 2004 .

[38]  G. Bruce Swedish word accents in sentence perspective , 1977 .

[39]  W. Cooper,et al.  Acoustical aspects of contrastive stress in question-answer contexts. , 1985, The Journal of the Acoustical Society of America.

[40]  David Bradley,et al.  Prosodic analysis and asian linguistics : to honour R. K. Sprigg , 1989 .

[41]  Chilin Shih,et al.  Comparison of Declarative and Interrogative Intonation in Chinese , 2002 .

[42]  Carrie S. Masek,et al.  Papers from the Parasession on Language and Behavior, Chicago Linguistic Society, May 1-2, 1981 , 1981 .

[43]  Emily Q. Wang,et al.  Pitch targets and their realization: Evidence from Mandarin Chinese , 2001, Speech Commun..

[44]  N. Thorsen,et al.  A study of perception of sentence intonation--evidence from Danish. , 1980, The Journal of the Acoustical Society of America.

[45]  Shunde Jin,et al.  An acoustic study of sentence stress in Mandarin Chinese , 1996 .

[46]  Klaus R. Scherer,et al.  Vocal communication of emotion: A review of research paradigms , 2003, Speech Commun..

[47]  D. Ladd,et al.  Stability of tonal alignment: the case of Greek prenuclear accents , 1998 .

[48]  Dwight L. Bolinger,et al.  Intonation and Its Uses: Melody in Grammar and Discourse , 1989 .

[49]  Yi Xu,et al.  Effects of tone and focus on the formation and alignment of f0contours , 1999 .

[50]  Yi Xu,et al.  On the Temporal Domain of Focus , 2004 .

[51]  W. Zemlin Speech and Hearing Science : Anatomy and Physiology , 1968 .

[52]  Chilin Shih,et al.  Hierarchical Structure and Word Strength Prediction of Mandarin Prosody , 2003, Int. J. Speech Technol..

[53]  J. Ohala,et al.  An Ethological Perspective on Common Cross-Language Utilization of F₀ of Voice , 1984, Phonetica.

[54]  Carlos Gussenhoven,et al.  Intonation and interpretation: phonetics and phonology , 2002, Speech Prosody 2002.

[55]  Yi Xu,et al.  Phonetic realization of focus in English declarative intonation , 2005, J. Phonetics.

[56]  I. Lehiste The Phonetic Structure of Paragraphs , 1975 .

[57]  M. Slowiaczek,et al.  Prosodic structure in language understanding: evidence from tone sandhi in Mandarin. , 1989, Language and speech.

[58]  Scott Myers The Effects of Boundary Tones on the f 0 Scaling of Lexical Tones , 2004 .

[59]  Ching X. Xu,et al.  Effects of consonant aspiration on Mandarin tones , 2001, Journal of the International Phonetic Association.

[60]  A Antonie Cohen Perceptual analysis of intonation patterns , 1965 .

[61]  Yi Xu,et al.  Contextual tonal variation in Mandarin Chinese , 1993 .

[62]  J Vincent,et al.  What is the smallest prosodic domain , 1994 .

[63]  J. Pierrehumbert The phonology and phonetics of English intonation , 1987 .

[64]  Yi Xu Haskins THE PENTA MODEL OF SPEECH MELODY : TRANSMITTING MULTIPLE COMMUNICATIVE FUNCTIONS IN PARALLEL , 2004 .

[65]  R. Krakow Physiological organization of syllables: a review , 1999 .

[66]  Chilin Shih,et al.  The prosodic domain of tone sandhi in Chinese , 1986 .

[67]  Scott Myers DOWNDRIFT AND PITCH RANGE IN CHICHEWA INTONATION , 1999 .

[68]  J. Masefield,et al.  ‘ ‘ Sagging transitions ’ ’ between high pitch accents in English : experimental evidence , 2003 .

[69]  Rpg Rene Collier,et al.  Focus Conditions and the Prominence of Pitch-Accented Syllables , 1996 .

[70]  Philip Rose On the non-equivalence of fundamental frequency and pitch in tonal description , 1988 .

[71]  Yi Xu,et al.  Separation of functional components of tone and intonation from observed F0 patterns , 2004 .

[72]  W S Wang,et al.  Tone 3 in Pekinese. , 1967, Journal of speech and hearing research.

[73]  J. Pierrehumbert,et al.  Japanese Tone Structure , 1988 .

[74]  D. Ladd,et al.  Phonological conditioning of peak alignment in rising pitch accents in Dutch. , 2000, The Journal of the Acoustical Society of America.

[75]  S. Mozziconacci Prosody and emotions , 2002, Proceedings of the International Conference on Speech Prosody.

[76]  Robert Bannert,et al.  Contrastive Tonal Analysis of Focus Perception in Greek and Swedish , 2000 .

[77]  L. Angeles The Effects of Duration and Sonority on Contour Tone Distribution— Typological Survey and Formal Analysis , 2001 .

[78]  C. Shih,et al.  A Declination Model of Mandarin Chinese , 2000 .

[79]  Vicky C. H. Man Focus Effects on Cantonese Tones: An Acoustic Study , 2002 .

[80]  Hansjörg Mixdorff,et al.  Quantitative Tone and Intonation Modeling across Languages , 2004 .

[81]  R. Collier,et al.  Declination: Construct or Intrinsic Feature of Speech Pitch ? , 1982, Phonetica.

[82]  Y Xu,et al.  Consistency of Tone-Syllable Alignment across Different Syllable Structures and Speaking Rates , 1998, Phonetica.

[83]  Anne Wichmann,et al.  THE ATTITUDINAL EFFECTS OF PROSODY , AND HOW THEY RELATE TO EMOTION , 2000 .

[84]  K. Hata,et al.  Fundamental Frequency as an Acoustic Cue to Accent Perception , 1992, Language and speech.

[85]  X. Shen,et al.  Mandarin neutral tone revisited , 1992 .

[86]  D. Bolinger Intonation and its parts : melody in spoken English , 1987 .

[87]  Shirley A. Steele Nuclear accent F0 peak location: Effects of rate, vowel, and number of following syllables , 1986 .

[88]  Daniel Hirst,et al.  Automatic modelling of fundamental frequency using a quadratic sline function , 1993 .

[89]  John J. Ohala,et al.  Speed of Pitch Change , 1973 .

[90]  Maolin Wang,et al.  Tonal and durational variations as phonetic coding for syllable grouping , 2005 .

[91]  Y Xu,et al.  Production and perception of coarticulated tones. , 1994, The Journal of the Acoustical Society of America.

[92]  D. Ladd,et al.  Declination.: a review and some hypotheses , 1984, Phonology Yearbook.

[93]  Hiroya Fujisaki,et al.  Dynamic Characteristics of Voice Fundamental Frequency in Speech and Singing , 1983 .

[94]  Yi Xu,et al.  Fundamental Frequency Peak Delay in Mandarin , 2000, Phonetica.

[95]  Chilin Shih,et al.  Quantitative measurement of prosodic strength in Mandarin , 2003, Speech Commun..

[96]  K. Pike,et al.  The intonation of American English , 1946 .

[97]  M. Gordon Syllable Weight: Phonetics, Phonology, Typology , 2006 .

[98]  Yi Xu,et al.  A pitch target approximation model for F0 contours in Mandarin , 1999 .

[99]  John Kingston,et al.  Papers in Laboratory Phonology: Index of names , 1990 .

[100]  Yi Xu,et al.  Multiple effects of consonant manner of articulation and intonation type on F0 in English , 2004 .

[101]  A. B.,et al.  SPEECH COMMUNICATION , 2001 .

[102]  W. L. Nelson Physical principles for economies of skilled movements , 1983, Biological Cybernetics.

[103]  Harry Hollien,et al.  Vocal Pitch Variation Related to Changes in Vocal Fold Length , 1960 .

[104]  趙 元任,et al.  A grammar of spoken Chinese = 中國話的文法 , 1968 .

[105]  P. MacNeilage,et al.  The frame/content theory of evolution of speech production , 1998, Behavioral and Brain Sciences.

[106]  Jessica Dawn Cox,et al.  Of This Place , 2007 .

[107]  Yi Xu,et al.  Compensation for pitch-shifted auditory feedback during the production of Mandarin tone sequences. , 2004, The Journal of the Acoustical Society of America.

[108]  Yi Xu,et al.  Asking questions with focus , 2004 .

[109]  X. Shen The Prosody of Mandarin Chinese , 1990 .

[110]  Tan Lee,et al.  0 ANALYSIS OF CONTINUOUS CANTONESE SPEECH , 2002 .

[111]  J. Pierrehumbert,et al.  Synthesizing intonation , 2004 .

[112]  Lin Maocan Tonal coarticulation patterns in quadrisyllabie word and phrase of mandarin , 1992 .

[113]  N. Umeda F0 Declination is situation dependent , 1980 .

[114]  W. Cooper,et al.  Speech intonation and focus location in matched statements and questions. , 1986, The Journal of the Acoustical Society of America.

[115]  P Taylor,et al.  Analysis and synthesis of intonation using the Tilt model. , 2000, The Journal of the Acoustical Society of America.

[116]  Osamu Fujimura,et al.  The C/D Model and Prosodic Control of Articulatory Behavior , 2000, Phonetica.

[117]  Elisabeth Selkirk Prosodic Evidence from Right Node Raising in English * , 2002 .

[118]  M. Turvey,et al.  Phase transitions and critical fluctuations in the visual coordination of rhythmic movements between people. , 1990, Journal of experimental psychology. Human perception and performance.

[119]  D. Robert Ladd,et al.  On the phonetics and phonology of "segmental anchoring" of F0: evidence from German , 2004, J. Phonetics.

[120]  G. P. Moore,et al.  Measurements of the Vocal Folds during Changes in Pitch , 1960 .

[121]  M. Swerts Prosodic features at discourse boundaries of different strength. , 1997, The Journal of the Acoustical Society of America.

[122]  D. Bolinger Accent Is Predictable (If You're a Mind-Reader) , 1972 .

[123]  E Gårding Swedish prosody. Summary of a project. , 1982, Phonetica.

[124]  Mariapaola D’Imperio Italian intonation: An overview and some questions , 2002 .

[125]  Emiel Krahmer,et al.  On the alleged existence of contrastive accents , 2001, Speech Commun..

[126]  D. Fry Experiments in the Perception of Stress , 1958 .

[127]  Yi Xu,et al.  Understanding tone from the perspective of production and perception. Language and Linguistics , 2004 .

[128]  M. Horne Prosody, theory and experiment : studies presented to Gösta Bruce , 2000 .

[129]  Paavo Alku,et al.  Measuring the effect of fundamental frequency raising as a strategy for increasing vocal intensity in soft, normal and loud phonation , 2002, Speech Commun..

[130]  Alan W. Black,et al.  Generating F/sub 0/ contours from ToBI labels using linear regression , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[131]  X. Shen On tone sandhi and tonal coarticulation , 1992 .

[132]  Antonis Botinis Intonation: Analysis, Modelling and Technology , 2000 .

[133]  Charles Speaks,et al.  Journal of Speech and Hearing Research , 1963 .

[134]  Siripong Potisuk,et al.  Tonal Coarticulation in Thai , 1994 .

[135]  Yetunde Ọlabisi Laniran,et al.  Intonation in tone languages : the phonetic implementation of tones in Yorùbá , 1992 .

[136]  Roddy Cowie,et al.  Describing the emotional states that are expressed in speech , 2003, Speech Commun..

[137]  D H Whalen,et al.  Perception of pitch location within a speaker's F0 range. , 2005, The Journal of the Acoustical Society of America.

[138]  B. G. Fivela,et al.  Tonal alignment in two pisa Italian peak accents , 2002, Speech Prosody 2002.

[139]  Fernand Carton L'accent d'insistance = Emphatic stress , 1976 .

[140]  Mariapaola D'Imperio,et al.  Focus and tonal structure in Neapolitan Italian , 2001, Speech Commun..