Building an automated English sentence evaluation system for students learning English as a second language

This paper presents an automated scoring system which grades students' English writing tests. The system provides a score and diagnostic feedback to students without human's efforts. Target users are Korean students in junior high schools who learn English as a second language. The system takes a single English sentence as its input. Dealing with a single sentence as an input has some advantages on comparing the input with the answers given by human teachers and giving detailed feedback to the students. The system was developed and tested with the real test data collected through English tests given to third grade students in junior high school. Scoring requires two steps of the process. The first process is analyzing the input sentence in order to detect possible errors, such as spelling errors and syntactic errors. The second process is comparing the input sentence with given answers to identify the differences as errors. To evaluate the performance of the system, the output produced by the system is compared with the result provided by human raters. The score agreement value between a human rater and the system is quite close to the value between two human raters.

[1]  Jianfeng Gao,et al.  Using Contextual Speller Techniques and Language Modeling for ESL Error Correction , 2008, IJCNLP.

[2]  Jill Burstein,et al.  Advanced Capabilities for Evaluating Student Writing: Detecting Off-Topic Essays Without Topic-Specific Training , 2005, AIED.

[3]  Rachele De Felice,et al.  A Classifier-Based Approach to Preposition and Determiner Error Correction in L2 English , 2008, COLING.

[4]  Louisa Sadler,et al.  Structural Non-Correspondence in Translation , 1991, EACL.

[5]  Rada Mihalcea,et al.  Text-to-Text Semantic Similarity for Automatic Short Answer Grading , 2009, EACL.

[6]  D. Crystal,et al.  English as a Global Language , 1998 .

[7]  Stephanie Seneff,et al.  Automatic grammar correction for second-language learners , 2006, INTERSPEECH.

[8]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[9]  Martin Chodorow,et al.  Automated Essay Evaluation: The Criterion Online Writing Service , 2004, AI Mag..

[10]  Johanna D. Moore,et al.  Using Natural Language Processing to Analyze Tutorial Dialogue Corpora Across Domains Modalities , 2009, AIED.

[11]  Martin Chodorow,et al.  An Unsupervised Method for Detecting Grammatical Errors , 2000, ANLP.

[12]  Daniel Marcu,et al.  Finding the WRITE Stuff: Automatic Identification of Discourse Structure in Student Essays , 2003, IEEE Intell. Syst..

[13]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[14]  Kong Joo Lee,et al.  A Human-Computer Collaboration Approach to Improve Accuracy of an Automated English Scoring System , 2010 .

[15]  Mirella Lapata,et al.  Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008) , 2008 .