Using Entity-Based Features to Model Coherence in Student Essays

We show how the Barzilay and Lapata entity-based coherence algorithm (2008) can be applied to a new, noisy data domain --- student essays. We demonstrate that by combining Barzilay and Lapata's entity-based features with novel features related to grammar errors and word usage, one can greatly improve the performance of automated coherence prediction for student essays for different populations.

[1]  Viviana Cortes,et al.  Shell nouns as cohesive devices in published and ESL student writing , 2008 .

[2]  Marti A. Hearst.  Text Tiling: Segmenting Text into Multi-paragraph Subtopic Passages , 1997, CL.

[3]  Daniel Marcu,et al.  Evaluating Multiple Aspects of Coherence in Student Essays , 2004, NAACL.

[4]  Martin Chodorow,et al.  Rethinking Grammatical Error Annotation and Evaluation with the Amazon Mechanical Turk , 2010 .

[5]  Claudia Leacock,et al.  Automated Grammatical Error Detection for Language Learners , 2010, Synthesis Lectures on Human Language Technologies.

[6]  Peter W. Foltz,et al.  The Measurement of Textual Coherence with Latent Semantic Analysis. , 1998 .

[7]  Ani Nenkova,et al.  Revisiting Readability: A Unified Framework for Predicting Text Quality , 2008, EMNLP.

[8]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[9]  Xiaoming Xi,et al.  A three-stage approach to the automated scoring of spontaneous spoken responses , 2011, Comput. Speech Lang..

[10]  Scott Weinstein,et al.  Centering: A Framework for Modeling the Local Coherence of Discourse , 1995, CL.

[11]  Karen Kukich,et al.  Automated Evaluation of Coherence in Student Essays , .

[12]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[13]  Xiaoming Xi,et al.  Towards Using Structural Events To Assess Non-native Speech , 2010 .

[14]  Jennifer Foster,et al.  Using Parse Features for Preposition Selection and Error Detection , 2010, ACL.

[15]  Joel R. Tetreault,et al.  The utility of article and preposition error correction systems for English language learners: Feedback and assessment , 2010 .

[16]  Randy Elliot,et al.  Automated Scoring of Constructed-Response Literacy and Mathematics Items , 2011 .

[17]  Klaus Zechner,et al.  Using Amazon Mechanical Turk for Transcription of Non-Native Speech , 2010, Mturk@HLT-NAACL.

[18]  Na-Rae Han,et al.  Using an Error-Annotated Learner Corpus to Develop an ESL/EFL Error Correction System , 2010, LREC.

[19]  Martin Chodorow,et al.  Progress and New Directions in Technology for Automated Essay Evaluation , 2010 .