ASAP++: Enriching the ASAP Automated Essay Grading Dataset with Essay Attribute Scores

In this paper, we describe the creation of a resource ASAP++ which is basically annotations of the Automatic Student Assessment Prize’s Automatic Essay Grading dataset. These annotations are scores for different attributes of the essays, such as content, word choice, organization, sentence fluency, etc. Each of these essays is scored by an annotator. We also report the results of each of the attributes using a Random Forest Classifier using a baseline set of attribute independent features as described by Zesch et al. (2015). We release and share this resource to facilitate further research into these attributes of essay grading.

[1]  Yue Zhang,et al.  Automatic Features for Essay Scoring – An Empirical Study , 2016, EMNLP.

[2]  Mirella Lapata,et al.  Modeling Local Coherence: An Entity-Based Approach , 2005, ACL.

[3]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[4]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[5]  Hwee Tou Ng,et al.  A Neural Approach to Automated Essay Scoring , 2016, EMNLP.

[6]  Jill Burstein,et al.  Handbook of Automated Essay Evaluation Current Applications and New Directions , 2018 .

[7]  William Wresch,et al.  The Imminence of Grading Essays by Computer-25 Years Later , 1993 .

[8]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[9]  Vincent Ng,et al.  Modeling Prompt Adherence in Student Essays , 2014, ACL.

[10]  Helen Yannakoudakis,et al.  Automatic Text Scoring Using Neural Networks , 2016, ACL.

[11]  Hwee Tou Ng,et al.  Flexible Domain Adaptation for Automated Essay Scoring Using Correlated Linear Regression , 2015, EMNLP.

[12]  Eric P. Xing,et al.  Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2014, ACL 2014.

[13]  Yue Zhang,et al.  Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring , 2017, CoNLL.

[14]  Swapna Somasundaran,et al.  Lexical Chaining for Measuring Discourse Coherence Quality in Test-taker Essays , 2014, COLING.

[15]  Torsten Zesch,et al.  Task-Independent Features for Automated Essay Grading , 2015, BEA@NAACL-HLT.

[16]  Vincent Ng,et al.  Modeling Organization in Student Essays , 2010, EMNLP.

[17]  Ben He,et al.  Automated Essay Scoring by Maximizing Human-Machine Agreement , 2013, EMNLP.

[18]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .