Towards Using Structural Events To Assess Non-native Speech

We investigated using structural events, e.g., clause and disfluency structure, from transcriptions of spontaneous non-native speech, to compute features for measuring speaking proficiency. Using a set of transcribed audio files collected from the TOEFL Practice Test Online (TPO), we conducted a sophisticated annotation of structural events, including clause boundaries and types, as well as disfluencies. Based on words and the annotated structural events, we extracted features related to syntactic complexity, e.g., the mean length of clause (MLC) and dependent clause frequency (DEPC), and a feature related to disfluencies, the interruption point frequency per clause (IPC). Among these features, the IPC shows the highest correlation with holistic scores (r = -0.344). Furthermore, we increased the correlation with human scores by normalizing IPC by (1) MLC (r = -0.386), (2) DEPC (r = -0.429), and (3) both (r = -0.462). In this research, the features derived from structural events of speech transcriptions are found to predict holistic scores measuring speaking proficiency. This suggests that structural events estimated on speech word strings provide a potential way for assessing non-native speech.

[1]  Mary P. Harper,et al.  Structural event detection for rich transcription of speech , 2004 .

[2]  Gökhan Tür,et al.  Prosody-based automatic segmentation of speech into sentences and topics , 2000, Speech Commun..

[3]  K. W. Hunt.  Syntactic maturity in schoolchildren and adults. , 1970, Monographs of the Society for Research in Child Development.

[4]  Su-Youn Yoon.  Automated Assessment of Speech Fluency for L2 English Learners , 2009 .

[5]  Noriko Iwashita,et al.  Syntactic Complexity Measures and Their Relation to Oral Proficiency in Japanese as a Foreign Language , 2006 .

[6]  L. Ortega.  Syntactic Complexity Measures and Their Relationship to L2 Proficiency: A Research Synthesis of College-Level L2 Writing. , 2003 .

[7]  Mitch Weintraub,et al.  Automatic scoring of pronunciation quality , 2000, Speech Commun..

[8]  Alan Tonkyn,et al.  Measuring spoken language: a unit for all reasons , 2000 .

[9]  P. Lennon.  Investigating Fluency in EFL: A Quantitative Approach* , 1990 .

[10]  G. Mizera,et al.  WORKING MEMORY AND L2 ORAL FLUENCY , 2006 .

[11]  Johan Frid,et al.  Measuring Syntactic Complexity in Spontaneous Spoken Swedish , 2007, Language and speech.

[12]  Betty Schrampfer Azar,et al.  Fundamentals of English Grammar , 1985 .

[13]  Dilek Z. Hakkani-Tür,et al.  Speech segmentation and spoken document processing , 2008, IEEE Signal Processing Magazine.

[14]  Xiaoming Xi,et al.  SpeechraterTM: a construct-driven approach to scoring spontaneous non-native speech , 2007, SLaTE.