A weighted reliability measure for phonetic transcription.

PURPOSE The purpose of the present work is to describe and illustrate the utility of a new tool for assessment of transcription agreement. Traditional measures have not characterized overall transcription agreement with sufficient resolution, specifically because they have often treated all phonetic differences between segments in transcriptions as equivalent, thus constituting an unweighted approach to agreement assessment. The measure the authors have developed calculates a weighted transcription agreement value based on principles derived from widely accepted tenets of phonological theory. METHOD To investigate the utility of the new measure, 8 coders transcribed samples of speech and infant vocalizations. Comparing the transcriptions through a computer-based implementation of the new weighted and the traditional unweighted measures, they investigated the scaling properties of both. RESULTS The results illustrate better scaling with the weighted measure, in particular because the weighted measure is not subject to the floor effects that occur with the traditional measure when applied to samples that are difficult to transcribe. Furthermore, the new weighted measure shows orderly relations in degree of agreement across coded samples of early canonical-stage babbling, early meaningful speech in English, and 3 adult languages. CONCLUSIONS The authors conclude that the weighted measure may provide improved foundations for research on phonetic transcription and for monitoring of transcription reliability.

[1]  J. P. Sutcliffe,et al.  Error of measurement and the sensitivity of a test of significance , 1958 .

[2]  I. Stockman,et al.  Listener agreement on phonetic segments in early infant vocalizations , 1981, Journal of psycholinguistic research.

[3]  P. MacNeilage,et al.  The articulatory basis of babbling. , 1995, Journal of speech and hearing research.

[4]  Florien J. Koopmans-van Beinum,et al.  Early Stages in the Development of Speech Movements , 1986 .

[5]  J. Firth,et al.  Papers in linguistics, 1934-1951 , 1957 .

[6]  Bruno D. Zumbo,et al.  Reliability of Measurement and Power of Significance Tests Based on Differences , 1993 .

[7]  John J. McCarthy,et al.  Formal Problems in Semitic Phonology and Morphology , 2018 .

[8]  Rebecca E. Eilers,et al.  Phonetic Expectation and Transcription Validity , 1975, Phonetica.

[9]  D. Oller.  THE EMERGENCE OF THE SOUNDS OF SPEECH IN INFANCY , 1980 .

[10]  D. Oller,et al.  Infant babbling and speech , 1976, Journal of Child Language.

[11]  K. Pollock,et al.  An analysis of the vowel misarticulations of five children with developmental apraxia of speech , 1991 .

[12]  J. B. Delack.  Aspects of infant speech development in the first year of life , 1976, Canadian Journal of Linguistics/Revue canadienne de linguistique.

[13]  C. Cucchiarini.  Assessing transcription agreement: Methodological aspects , 1996 .

[14]  Barbara L. Davis,et al.  Early Vocal Patterns in Infants with Varied Hearing Levels , 2005 .

[15]  M. Edwards,et al.  Issues in Collecting and Transcribing Speech Samples. , 2001 .

[16]  O. C. Irwin,et al.  A reliability study of speech sounds observed in the crying of newborn infants , 1941 .

[17]  B. Hayes.  Metrical Stress Theory: Principles and Case Studies , 1995 .

[18]  Marlys A. Macken,et al.  From Babbling to Speech: A Re-Assessment of the Continuity Issue , 1985 .

[19]  Rachel E. Stark,et al.  Chapter 5 – RETRACTED: STAGES OF SPEECH DEVELOPMENT IN THE FIRST YEAR OF LIFE1 , 1980 .

[20]  G. A. Miller,et al.  An Analysis of Perceptual Confusions Among Some English Consonants , 1955 .

[21]  R Plomp,et al.  Objective analysis versus subjective assessment of vowels pronounced by deaf and normal-hearing children. , 1995, The Journal of the Acoustical Society of America.

[22]  W. Alan Nicewander,et al.  Reliability of measurement and the power of statistical tests: Some new results. , 1983 .

[23]  P. Ladefoged.  A course in phonetics , 1975 .

[24]  D. Oller,et al.  A Pragmatic Approach to Phonological Systems of Deaf Speakers , 1981 .

[25]  C. Stoel-Gammon,et al.  Patterns of early lexical and phonological development , 1984, Journal of Child Language.

[26]  D. Ingram.  The measurement of whole-word productions , 2002, Journal of Child Language.

[27]  Carol Stoel-Gammon,et al.  Transcribing the Speech of Young Children. , 2001 .

[28]  Raymond D. Kent,et al.  Toward phonetic intelligibility testing in dysarthria. , 1989, The Journal of speech and hearing disorders.

[29]  M. Halle,et al.  Preliminaries to Speech Analysis: The Distinctive Features and Their Correlates , 1961 .

[30]  Richard M. Warren,et al.  A comparison of speech perception in childhood, maturity, and old age by means of the verbal transformation effect , 1966 .

[31]  D K Oller,et al.  Similarity of babbling in Spanish- and English-learning babies , 1982, Journal of Child Language.

[32]  D K Oller,et al.  Speech-like vocalizations in infancy: an evaluation of potential risk factors , 1994, Journal of Child Language.

[33]  C. Smith,et al.  Residual hearing and speech production in deaf children. , 1975, Journal of speech and hearing research.

[34]  D. Bacon.  The Contributions of Reliability and Pretests to Effective Assessment , 2004 .

[35]  Nikolaj Sergejevič Trubeckoj.  Grundzüge der Phonologie , 1989 .

[36]  C. Stoel-Gammon,et al.  Vowel systems of normally developing and phonologically disordered children. , 1990, Clinical linguistics & phonetics.

[37]  D. Oller.  The emergence of the speech capacity , 2000 .

[38]  C. Stoel-Gammon,et al.  Babbling development of hearing-impaired and normally hearing subjects. , 1986, The Journal of speech and hearing disorders.

[39]  O. C. Irwin,et al.  Vowel elements in the crying vocalization of infants under ten days of age , 1941 .

[40]  Noam Chomsky,et al.  The Sound Pattern of English , 1968 .

[41]  P. Smolensky,et al.  Optimality Theory: Constraint Interaction in Generative Grammar , 2004 .

[42]  G. William Walster,et al.  Effect of Reliability and Validity on Power of Statistical Tests , 1970 .

[43]  D K Oller,et al.  The role of audition in infant babbling. , 1988, Child development.

[44]  D K Oller,et al.  Beyond ba-ba and gu-gu: Challenges and strategies in coding infant vocalizations , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[45]  D. Oller,et al.  The acquisition of voicing contrasts in Spanish and English learning infants and children: a longitudinal study , 1984, Journal of Child Language.

[46]  M. Legerstee.  Changes in the quality of infant sounds as a function of social and nonsocial stimulation , 1991 .

[47]  Marc Swerts,et al.  The Influence of the Labeller's Regional Background on Phonetic Transcriptions: Implications for the Evaluation of Spoken Language Resources , 2004, LREC.

[48]  L. Shriberg,et al.  Reliability studies in broad and narrow phonetic transcription , 1991 .

[49]  L D Shriberg,et al.  Developmental apraxia of speech: I. Descriptive and theoretical perspectives. , 1997, Journal of speech, language, and hearing research : JSLHR.