Automated Scoring of Nonnative Speech Using the  SpeechRater SM v. 5.0 Engine

[1]  Xiaoming Xi,et al.  Evaluating analytic scoring for the TOEFL® Academic Speaking Test (TAST) for operational use , 2007 .

[2]  Su-Youn Yoon,et al.  Non-English Response Detection Method for Automated Proficiency Scoring System , 2011, BEA@ACL.

[3]  Helmer Strik,et al.  My Pronunciation Coach: Computer-assisted English pronunciation training , 2014 .

[4]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[5]  Su-Youn Yoon,et al.  Application of Structural Events Detected on ASR Outputs for Automated Speaking Assessment , 2012, INTERSPEECH.

[6]  David M. Williamson,et al.  Automated essay scoring: Psychometric guidelines and practices , 2013 .

[7]  Klaus Zechner,et al.  Computing and Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-Native Speech , 2011, ACL.

[8]  Su-Youn Yoon,et al.  Vocabulary Profile as a Measure of Vocabulary Sophistication , 2012, BEA@NAACL-HLT.

[9]  Klaus Zechner,et al.  What did they actually say? agreement and disagreement among transcribers of non-native spontaneous speech responses in an English proficiency test , 2009, SLaTE.

[10]  Kristin Precoda,et al.  EduSpeak®: A speech recognition and pronunciation scoring toolkit for computer-aided language learning applications , 2010 .

[11]  Derrick Higgins,et al.  EVALUATING THE CONSTRUCT‐COVERAGE OF THE E‐RATER® SCORING ENGINE , 2009 .

[12]  Jian Cheng,et al.  Validating automated speaking tests , 2010 .

[13]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[14]  H. Riggenbach.  Toward an understanding of fluency: A microanalysis of nonnative speaker conversations , 1991 .

[15]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  Su-Youn Yoon,et al.  Automatic assessment of syntactic complexity for spontaneous speech scoring , 2015, Speech Commun..

[17]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[18]  Beata Beigman Klebanov,et al.  Content Importance Models for Scoring Writing From Sources , 2014, ACL.

[19]  G. Mizera,et al.  WORKING MEMORY AND L2 ORAL FLUENCY , 2006 .

[20]  F. Ramus,et al.  Correlates of linguistic rhythm in the speech signal , 1999, Cognition.

[21]  Helmer Strik,et al.  Web-based mini-games for language learning that support spoken interaction , 2015, SLaTE.

[22]  Anastassia Loukina,et al.  Automated Scoring of Nonnative Speech Using the  SpeechRater SM v. 5.0 Engine , 2018 .

[23]  Jian Cheng,et al.  Using deep neural networks to improve proficiency assessment for children English language learners , 2014, INTERSPEECH.

[24]  Yong Wang,et al.  Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiers , 2015, Speech Commun..

[25]  Anastassia Loukina,et al.  Feature selection for automated speech scoring , 2015, BEA@NAACL-HLT.

[26]  Elmar Nöth,et al.  Pronunciation Feature Extraction , 2005, DAGM-Symposium.

[27]  Maxine Eskénazi,et al.  An overview of spoken language technology for education , 2009, Speech Commun..

[28]  P. Lennon.  Investigating Fluency in EFL: A Quantitative Approach* , 1990 .

[29]  A. Vermeer.  Coming to grips with lexical richness in spontaneous speech data , 2000 .

[30]  Justin Zobel,et al.  Methods for Identifying Versioned and Plagiarized Documents , 2003, J. Assoc. Inf. Sci. Technol..

[31]  Klaus Zechner,et al.  Towards Automatic Scoring of Non-Native Spontaneous Speech , 2006, HLT-NAACL.

[32]  L. Boves,et al.  Quantitative assessment of second language learners' fluency: comparisons between read and spontaneous speech. , 2002, The Journal of the Acoustical Society of America.

[33]  Mary P. Harper,et al.  Structural event detection for rich transcription of speech , 2004 .

[34]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[35]  Keelan Evanini,et al.  The influence of automatic speech recognition accuracy on the performance of an automated speech assessment system , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[36]  Su-Youn Yoon,et al.  Assessment of ESL Learners' Syntactic Competence Based on Similarity Measures , 2012, EMNLP-CoNLL.

[37]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[38]  Xin Chen,et al.  Deep neural network acoustic models for spoken assessment applications , 2015, Speech Commun..

[39]  V. Dellwo.  Rhythm and Speech Rate: A Variation Coefficient for deltaC , 2006 .

[40]  Jian Cheng,et al.  Off-Topic Detection in Automated Speech Assessment Applications , 2011, INTERSPEECH.

[41]  Helmer Strik,et al.  Comparing different approaches for automatic pronunciation error detection , 2009, Speech Commun..

[42]  Jian Cheng,et al.  Fluency and structural complexity as predictors of L2 oral proficiency , 2010, INTERSPEECH.

[43]  W. Bruce Croft,et al.  Similarity measures for tracking information flow , 2005, CIKM '05.

[44]  Linda L. Cook,et al.  Fairness in Educational Assessment and Measurement , 2016 .

[45]  Helmer Strik,et al.  Automatic evaluation of Dutch pronunciation by using speech recognition technology , 1997, 1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings.

[46]  Joel R. Tetreault,et al.  Using Entity-Based Features to Model Coherence in Student Essays , 2010, HLT-NAACL.

[47]  Mark J. F. Gales,et al.  Automatically grading learners' English using a Gaussian process , 2015, SLaTE.

[48]  Shrikanth S. Narayanan,et al.  Connecting rhythm and prominence in automatic ESL pronunciation scoring , 2009, INTERSPEECH.

[49]  Anastassia Loukina,et al.  Automatic evaluation of spoken summaries: the case of language assessment , 2014, BEA@ACL.

[50]  Xiaoming Xi,et al.  Improved pronunciation features for construct-driven assessment of non-native spontaneous speech , 2009, HLT-NAACL.

[51]  Wai Kit Lo,et al.  Statistical phone duration modeling to filter for intact utterances in a computer-assisted pronunciation training system , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[52]  V. Dellwo,et al.  Comparing native and non-native speech rhythm using acoustic rhythmic measures: Cantonese, Beijing Mandarin and English , 2008 .

[53]  Helmer Strik,et al.  Modeling pronunciation variation for ASR: A survey of the literature , 1999, Speech Commun..

[54]  Xiaoming Xi,et al.  SpeechraterTM: a construct-driven approach to scoring spontaneous non-native speech , 2007, SLaTE.

[55]  Noriko Iwashita,et al.  Features of oral proficiency in task performance by EFL and JFL learners , 2010 .

[56]  Helmer Strik,et al.  Quantitative assessment of second language learners' fluency , 2000 .

[57]  Ronald A. Cole,et al.  The OGI multi-language telephone speech corpus , 1992, ICSLP.

[58]  Sid-Ahmed Selouani,et al.  Native and non-native class discrimination using speech rhythm- and auditory-based cues , 2015, Comput. Speech Lang..

[59]  Laurence White,et al.  Calibrating rhythm: First language and second language studies , 2007, J. Phonetics.

[60]  Klaus Zechner,et al.  Exploring Content Features for Automated Speech Scoring , 2012, HLT-NAACL.

[61]  Xiaofei Lu,et al.  Automatic analysis of syntactic complexity in second language writing , 2010 .

[62]  R. Hout,et al.  Lexical richness in the spontaneous speech of bilinguals , 2003 .

[63]  Klaus Zechner,et al.  Coherence Modeling for the Automated Assessment of Spontaneous Spoken Responses , 2013, NAACL.

[64]  Xiaoming Xi,et al.  Automatic scoring of non-native spontaneous speech in tests of spoken English , 2009, Speech Commun..

[65]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[66]  Nathan Schneider,et al.  Association for Computational Linguistics: Human Language Technologies , 2011 .

[67]  Regina Barzilay,et al.  Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization , 2004, NAACL.

[68]  Lei Chen,et al.  Applying Rhythm Features to Automatically Assess Non-Native Speech , 2011, INTERSPEECH.

[69]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[70]  Mitch Weintraub,et al.  Automatic scoring of pronunciation quality , 2000, Speech Commun..

[71]  Antony John Kunnan,et al.  The Duolingo English Test , 2015 .

[72]  M. Sanderson,et al.  Duplicate Detection in the Reuters Collection , 1997 .

[73]  Laurence White,et al.  Language categorization by adults is based on sensitivity to durational cues, not rhythm class , 2012 .

[74]  Peter W. Foltz,et al.  Textual coherence using latent semantic analysis , 1998 .

[75]  Liz Temple,et al.  Second language learner speech production , 2000 .

[76]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[77]  Lei He,et al.  Syllabic intensity variations as quantification of speech rhythm: evidence from both L1 and L2 , 2012, Speech Prosody 2012.

[78]  Frank K. Soong,et al.  The Use of DBN-HMMs for Mispronunciation Detection and Diagnosis in L2 English to Support Computer-Aided Pronunciation Training , 2012, INTERSPEECH.

[79]  J. Goeman.  L1 Penalized Estimation in the Cox Proportional Hazards Model , 2009, Biometrical journal. Biometrische Zeitschrift.

[80]  Larry S. Davis,et al.  Automated Scoring of Speaking Tasks in the Test of English‐for‐Teaching (TEFT™) , 2015 .

[81]  Arthur C. Graesser,et al.  Coh-Metrix: Analysis of text on cohesion and language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[82]  T. McNamara,et al.  Assessed Levels of Second Language Speaking Proficiency: How Distinct? , 2007 .

[83]  Carlos Gussenhoven,et al.  Durational variability in speech and the Rhythm Class Hypothesis , 2002 .

[84]  Xiaoming Xi,et al.  A three-stage approach to the automated scoring of spontaneous spoken responses , 2011, Comput. Speech Lang..

[85]  Vassilios Digalakis,et al.  Automatic pronunciation evaluation of foreign speakers using unknown text , 2007, Comput. Speech Lang..

[86]  Elmar Nöth,et al.  Automatic assessment of non-native prosody , 2009, SLaTE.

[87]  Ani Nenkova,et al.  Automatic Evaluation of Linguistic Quality in Multi-Document Summarization , 2010, ACL.

[88]  David M. Williamson,et al.  A Framework for Evaluation and Use of Automated Scoring , 2012 .

[89]  Kristin Precoda,et al.  The SRI EduSpeak System: Recognition and Pronunciation Scoring for Language Learning , 2007 .

[90]  D. S. Boomer.  Hesitation and Grammatical Encoding , 1965, Language and speech.

[91]  Klaus Zechner,et al.  Applying rhythm metrics to non-native spontaneous speech , 2013, SLaTE.

[92]  Anastassia Loukina,et al.  Rhythm measures and dimensions of durational variation in speech. , 2011, The Journal of the Acoustical Society of America.

[93]  Kathryn Bock,et al.  Language production : Grammatical encoding , 1994 .

[94]  Klaus Zechner,et al.  Prompt-based Content Scoring for Automated Spoken Language Assessment , 2013, BEA@NAACL-HLT.

[95]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[96]  Frank K. Soong,et al.  A DNN-based acoustic modeling of tonal language and its application to Mandarin pronunciation training , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[97]  Lei Chen,et al.  Detecting Structural Events for Assessing Non-Native Speech , 2011, BEA@ACL.

[98]  Xiaoming Xi,et al.  Towards Automatic Scoring of a Test of Spoken Language with Heterogeneous Task Types , 2008 .

[99]  L. Ortega.  Syntactic Complexity Measures and Their Relationship to L2 Proficiency: A Research Synthesis of College-Level L2 Writing. , 2003 .

[100]  Shunji Inagaki,et al.  Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity , 1998 .

[101]  Xiaoming Xi,et al.  Automated Scoring of Spontaneous Speech Using SpeechRater? v1.0. Research Report. ETS RR-08-62. , 2008 .

[102]  Marc A. Zissman,et al.  Comparison of : Four Approaches to Automatic Language Identification of Telephone Speech , 2004 .

[103]  Daniel Marcu,et al.  Evaluating Multiple Aspects of Coherence in Student Essays , 2004, NAACL.

[104]  Helen Yannakoudakis,et al.  Modeling coherence in ESOL learner texts , 2012, BEA@NAACL-HLT.

[105]  Yang Liu,et al.  Coherence in child language narratives: a case study of annotation and automatic prediction of coherence , 2012, WOCCI.

[106]  P. Nation,et al.  Vocabulary size and use: Lexical richness in L2 written production , 1995 .