Creating Corpora for Research in Feedback Comment Generation

In this paper, we report on datasets that we created for research in feedback comment generation — a task of automatically generating feedback comments such as a hint or an explanatory note for writing learning. There has been almost no such corpus open to the public and accordingly there has been a very limited amount of work on this task. In this paper, we first discuss the principle and guidelines for feedback comment annotation. Then, we describe two corpora that we have manually annotated with feedback comments (approximately 50,000 general comments and 6,700 on preposition use). A part of the annotation results is now available on the web, which will facilitate research in feedback comment generation

[1]  Raymond Hendy Susanto,et al.  The CoNLL-2014 Shared Task on Grammatical Error Correction , 2014 .

[2]  Shin'ichiro Ishikawa,et al.  The ICNALE and Sophisticated Contrastive Interlanguage Analysis of Asian Learners of English , 2013 .

[3]  Jason Chang,et al.  TellMeWhy: Learning to Explain Corrective Feedback for Second Language Learners , 2019, EMNLP/IJCNLP.

[4]  Kiyotaka Uchimoto,et al.  The NICT JLE Corpus Exploiting the language learners' speech database for research and education , 2004 .

[5]  Hwee Tou Ng,et al.  The CoNLL-2013 Shared Task on Grammatical Error Correction , 2013, CoNLL Shared Task.

[6]  Ryo Nagata,et al.  Toward a Task of Feedback Comment Generation for Writing Learning , 2019, EMNLP.

[7]  Joel R. Tetreault,et al.  JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correction , 2017, EACL.

[8]  Christopher D. Manning,et al.  Generating Typed Dependency Parses from Phrase Structure Parses , 2006, LREC.

[9]  Boris Katz,et al.  Universal Dependencies for Learner English , 2016, ACL.

[10]  Keisuke Sakaguchi,et al.  Phrase Structure Annotation and Parsing for Learner English , 2016, ACL.

[11]  Helen Yannakoudakis,et al.  A New Dataset and Method for Automatically Grading ESOL Texts , 2011, ACL.

[12]  Sylviane Granger,et al.  The International Corpus of Learner English , 1993 .

[13]  John Bitchener,et al.  The Effect of Different Types of Corrective Feedback on ESL Student Writing. , 2005 .

[14]  Erik Smitterberg,et al.  International Corpus of Learner English , 2004 .

[15]  Makoto Itami,et al.  Diagnostic Processing of Japanese for Computer-Assisted Second Language Learning , 2000, ACL.

[16]  Steven J. Ross,et al.  SALIENCE OF FEEDBACK ON ERROR AND ITS EFFECT ON EFL WRITING QUALITY , 1986 .

[17]  Dana R. Ferris,et al.  ERROR FEEDBACK IN L2 WRITING CLASSES: HOW EXPLICIT DOES IT NEED TO BE? , 2001 .

[18]  Edward W. D. Whittaker,et al.  Correcting Preposition Errors in Learner English Using Error Case Frames and Feedback Messages , 2014, ACL.

[19]  Kathleen F. McCoy,et al.  English Error Correction: A Syntactic User Model Based on Principled “Mal-Rule” Scoring , 1996 .

[20]  Yuji Matsumoto,et al.  The Effect of Learner Corpus Size in Grammatical Error Correction of ESL Writings , 2012, COLING.

[21]  Edward W. D. Whittaker,et al.  Creating a manually error-tagged and shallow-parsed learner corpus , 2011, ACL.