Quantitative, Notional, and Comprehensive Evaluations of Spontaneous Engaged Speech

This study provides specific evidence regarding the extent to which quantitative measures, common sense notional measures, and comprehensive measures adequately characterize spontaneous, although engaged, speech. As such, the study contributes to the growing body of literature describing the current limits of automatic systems for evaluating spoken proficiency, provides examples of the essential nature of various notional and comprehensive variables, supports continued development of hybrid systems, and includes suggestions for the possible utilization of additional variables for automatic analyses. Data for this study were gathered and analyzed as follows. After 4 weeks of activities related to career development, 20 native English speaking college freshmen made recordings in English explaining their career preferences. Three experiments were then conducted. Experiment 1 analyzed the recordings according to current quantitative analyses used in fully automatic evaluations of fluency. Experiment 2 examined the recordings through a perception study according to common everyday notions of fluency. Experiment 3 analyzed the recordings according to an adaptation of the comprehensive rubrics used by the Educational Testing Service (ETS) for evaluating oral proficiency. The comprehensive evaluation (Experiment 3) provided the most insight, and temporal quantitative measures (Experiment 1) provided the least insight concerning the proficiency of the 20 speakers.

[1]  David V. Canter,et al.  Temporal and Trait Facets of Personnel Assessment , 1990 .

[2]  Kartik Audhkhasi,et al.  Automatic Evaluation of Fluency in Spoken Language , 2009 .

[3]  Gillian Wigglesworth,et al.  Rater judgment and English language speaking proficiency , 2005 .

[4]  Louis Guttman,et al.  Smallest space analysis of intelligence and achievement tests. , 1969 .

[5]  Xiaoming Xi,et al.  INVESTIGATING THE CRITERION-RELATED VALIDITY OF THE TOEFL® SPEAKING SCORES FOR ITA SCREENING AND SETTING STANDARDS FOR ITAS , 2008 .

[6]  L. Boves,et al.  Quantitative assessment of second language learners' fluency: comparisons between read and spontaneous speech. , 2002, The Journal of the Acoustical Society of America.

[7]  Garry Molholt.  Computer-Assisted Instruction in Pronunciation for Chinese Speakers of American English , 1988 .

[8]  Helmer Strik,et al.  AUTOMATIC ASSESSMENT OF SECOND LANGUAGE LEARNERS' FLUENCY , 1999 .

[9]  Yusuke Kondo,et al.  Fundamental Research on Automatic Speech Evaluation of L 2 English , 2008 .

[10]  Dov Elizur,et al.  Facets of Personal Values: A Structural Analysis of Life and Work Values , 1999 .

[11]  Xiaoming Xi,et al.  SpeechraterTM: a construct-driven approach to scoring spontaneous non-native speech , 2007, SLaTE.

[12]  Meliss Holland,et al.  The Path of Speech Technologies in Computer-Assisted Language Learning , 2007 .

[13]  Vassilios Digalakis,et al.  Combination of machine scores for automatic grading of pronunciation quality , 2000, Speech Commun..

[14]  T. Miller,et al.  Delivery , 2013, Pest management science.

[15]  Matthew P. Aylett.  Disfluency and speech recognition profile factors , 2003, DiSS.

[16]  L. Boves,et al.  Quantitative assessment of second language learners' fluency by means of automatic speech recognition technology. , 2000, The Journal of the Acoustical Society of America.

[17]  J. Richards,et al.  Longman Dictionary of Applied Linguistics , 1986 .

[18]  Ashish Verma,et al.  Automatic evaluation of spoken english fluency , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Heather Hilton,et al.  The link between vocabulary knowledge and spoken L2 fluency , 2008 .

[20]  H. Branigan,et al.  Non-linguistic influences on rates of disfluency in spontaneous speech , 1999 .

[21]  Xiaoming Xi,et al.  Extracting meaningful speech features to support diagnostic feedback: an ECD approach to automated scoring , 2006 .

[22]  Charles W. Stansfield.  Technology and language testing : a collection of papers from the Seventh Annual Language Testing Research Colloquium, held at Educational Testing Service, Princeton, New Jersey, April 6-9, 1985 , 1986 .

[23]  Helmer Strik,et al.  The Pedagogy-Technology Interface in Computer Assisted Pronunciation Training , 2002 .

[24]  Helmer Strik,et al.  Feedback in computer assisted pronunciation training: technology push or demand pull? , 2002, INTERSPEECH.

[25]  Cristina Pardo-Ballester,et al.  Measuring Oral Proficiency in Distance, Face-to-Face, and Blended Classrooms , 2008 .

[26]  Louis Guttman,et al.  Facet Theory, Smallest Space Analysis, and Factor Analysis , 1982 .