Improving Argument Mining in Student Essays by Learning and Exploiting Argument Indicators versus Essay Topics

Argument mining systems for student essays need to be able to reliably identify argument components independently of particular essay topics. Thus in addition to features that model argumentation through topic-independent linguistic indicators such as discourse markers, features that can abstract over lexical signals of particular essay topics might also be helpful to improve performance. Prior argument mining studies have focused on persuasive essays and proposed a variety of largely lexicalized features. Our current study examines the utility of such features, proposes new features to abstract over the domain topics of essays, and conducts evaluations using both 10-fold cross validation as well as cross-topic validation. Experimental results show that our proposed features significantly improve argument mining performance in both types of cross-fold evaluation settings. Feature ablation studies further shed light on relative feature utility.

[1]  A. Knott,et al.  Using Linguistic Phenomena to Motivate a Set of Coherence Relations. , 1994 .

[2]  Regina Barzilay,et al.  Using Lexical Chains for Text Summarization , 1997 .

[3]  J. Carthy,et al.  TOPIC DETECTION , A NEW APPLICATION FOR LEXICAL CHAINING ? , 2000 .

[4]  Marc Moens,et al.  Articles Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status , 2002, CL.

[5]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[6]  Daniel Marcu,et al.  Finding the WRITE Stuff: Automatic Identification of Discourse Structure in Student Essays , 2003, IEEE Intell. Syst..

[7]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[8]  Martin Chodorow,et al.  Automated Essay Evaluation: The Criterion Online Writing Service , 2004, AI Mag..

[9]  Marie-Francine Moens,et al.  Automatic detection of arguments in legal texts , 2007, ICAIL.

[10]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[11]  Marie-Francine Moens,et al.  Study on the Structure of Argumentation in Case Law , 2008, JURIX.

[12]  Ani Nenkova,et al.  Automatic sense prediction for implicit discourse relations in text , 2009, ACL.

[13]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[14]  George E. Newell,et al.  Teaching and Learning Argumentative Reading and Writing: A Review of Research , 2011, Reading Research Quarterly.

[15]  Nitin Madnani,et al.  Identifying High-Level Organizational Elements in Argumentative Discourse , 2012, NAACL.

[16]  Dietrich Rebholz-Schuhmann,et al.  Automatic recognition of conceptualization zones in scientific articles and two life science applications , 2012, Bioinform..

[17]  Manfred Stede,et al.  From Argument Diagrams to Argumentation Mining in Texts: A Survey , 2013, Int. J. Cogn. Informatics Nat. Intell..

[18]  Noam Slonim,et al.  Context Dependent Claim Detection , 2014, COLING.

[19]  Jan Snajder,et al.  Back up your Stance: Recognizing Arguments in Online Discussions , 2014, ArgMining@ACL.

[20]  Diane J. Litman,et al.  Ontology-Based Argument Mining and Automatic Essay Scoring , 2014, ArgMining@ACL.

[21]  Claire Cardie,et al.  Identifying Appropriate Support for Propositions in Online User Comments , 2014, ArgMining@ACL.

[22]  Iryna Gurevych,et al.  Identifying Argumentative Discourse Structures in Persuasive Essays , 2014, EMNLP.

[23]  Iryna Gurevych,et al.  Annotating Argument Components and Relations in Persuasive Essays , 2014, COLING.

[24]  Lejian Liao,et al.  Shell Miner: Mining Organizational Phrases in Argumentative Texts in Social Media , 2014, 2014 IEEE International Conference on Data Mining.

[25]  Simone Teufel,et al.  Unsupervised learning of rhetorical structure with un-topic models , 2014, COLING.

[26]  Beata Beigman Klebanov,et al.  Applying Argumentation Schemes for Essay Scoring , 2014, ArgMining@ACL.

[27]  Diane J. Litman,et al.  Automatic Scoring of an Analytical Response-To-Text Assessment , 2014, Intelligent Tutoring Systems.

[28]  Kevin D. Ashley,et al.  Improving Science Writing in Research Methods Classes Through Computerized Argument Diagramming , 2015, CogSci.

[29]  Paolo Torroni,et al.  Context-Independent Claim Detection for Argument Mining , 2015, IJCAI.

[30]  Diane J. Litman,et al.  Extracting Argument and Domain Words for Identifying Argument Components in Texts , 2015, ArgMining@HLT-NAACL.