Coarse-grained Argumentation Features for Scoring Persuasive Essays

Scoring the quality of persuasive essays is an important goal of discourse analysis, addressed most recently with highlevel persuasion-related features such as thesis clarity, or opinions and their targets. We investigate whether argumentation features derived from a coarse-grained argumentative structure of essays can help predict essays scores. We introduce a set of argumentation features related to argument components (e.g., the number of claims and premises), argument relations (e.g., the number of supported claims) and typology of argumentative structure (chains, trees). We show that these features are good predictors of human scores for TOEFL essays, both when the coarsegrained argumentative structure is manually annotated and automatically predicted.

[1]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[2]  Iryna Gurevych,et al.  Annotating Argument Components and Relations in Persuasive Essays , 2014, COLING.

[3]  Rashmi Prasad,et al.  The Penn Discourse Treebank , 2004, LREC.

[4]  Diane J. Litman,et al.  Ontology-Based Argument Mining and Automatic Essay Scoring , 2014, ArgMining@ACL.

[5]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[6]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[7]  M. Chodorow,et al.  BEYOND ESSAY LENGTH: EVALUATING E-RATER®'S PERFORMANCE ON TOEFL® ESSAYS , 2004 .

[8]  Jill Burstein,et al.  The E-rater® scoring engine: Automated essay scoring with natural language processing. , 2003 .

[9]  Beata Beigman Klebanov,et al.  Applying Argumentation Schemes for Essay Scoring , 2014, ArgMining@ACL.

[10]  Martin Chodorow,et al.  Beyond Essay Length: Evaluating e-rater[R]'s Performance on TOEFL[R] Essays. Research Reports. Report 73. RR-04-04. , 2004 .

[11]  Martin Chodorow,et al.  TOEFL11: A CORPUS OF NON‐NATIVE ENGLISH , 2013 .

[12]  Vincent Ng,et al.  Modeling Argument Strength in Student Essays , 2015, ACL.

[13]  Iryna Gurevych,et al.  Parsing Argumentation Structures in Persuasive Essays , 2016, CL.

[14]  Iryna Gurevych,et al.  Identifying Argumentative Discourse Structures in Persuasive Essays , 2014, EMNLP.

[15]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[16]  Vincent Ng,et al.  Modeling Thesis Clarity in Student Essays , 2013, ACL.

[17]  Jill Burstein,et al.  AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0 , 2004 .

[18]  Noura Farra,et al.  Scoring Persuasive Essays Using Opinions and their Targets , 2015, BEA@NAACL-HLT.