Context-aware Argument Mining and Its Applications in Education

Context is crucial for identifying arguments and argumentative relations in text, but existing argument studies have not addressed context dependence adequately. In this thesis, we propose context-aware argument mining that makes use of contextual features extracted from writing topics and context sentences to improve state-of-the-art argument component and argumentative relation classifications. The effectiveness as well as generality of our proposed contextual features is proven through its application in different argument mining tasks in student essays. We further evaluate the applicability of our proposed argument mining models in automated persuasive essay scoring tasks.

[1]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[2]  Hwee Tou Ng,et al.  Flexible Domain Adaptation for Automated Essay Scoring Using Correlated Linear Regression , 2015, EMNLP.

[3]  Lejian Liao,et al.  Shell Miner: Mining Organizational Phrases in Argumentative Texts in Social Media , 2014, 2014 IEEE International Conference on Data Mining.

[4]  Daniel Marcu,et al.  Sentence Level Discourse Parsing using Syntactic and Lexical Information , 2003, NAACL.

[5]  Marie-Francine Moens,et al.  Argumentation mining: the detection, classification and structure of arguments in text , 2009, ICAIL.

[6]  Daniel Marcu,et al.  Building a Discourse-Tagged Corpus in the Framework of Rhetorical Structure Theory , 2001, SIGDIAL Workshop.

[7]  Owen Rambow,et al.  Identifying Justifications in Written Dialogs by Classifying Text as Argumentative , 2011, Int. J. Semantic Comput..

[8]  Andrew McCallum,et al.  Rethinking LDA: Why Priors Matter , 2009, NIPS.

[9]  Marie-Francine Moens,et al.  Study on the Structure of Argumentation in Case Law , 2008, JURIX.

[10]  Marie-Francine Moens,et al.  Argumentation mining , 2011, Artificial Intelligence and Law.

[11]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[12]  Klaus krippendorff,et al.  Measuring the Reliability of Qualitative Text Analysis Data , 2004 .

[13]  Graeme Hirst,et al.  Classifying arguments by scheme , 2011, ACL.

[14]  Iryna Gurevych,et al.  Argumentation Mining in Persuasive Essays and Scientific Articles from the Discourse Structure Perspective , 2014, ArgNLP.

[15]  Jimmy J. Lin,et al.  Generative Content Models for Structural Analysis of Medical Abstracts , 2006, BioNLP@NAACL-HLT.

[16]  Martin Chodorow,et al.  Automated Essay Evaluation: The Criterion Online Writing Service , 2004, AI Mag..

[17]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[18]  Naoaki Okazaki,et al.  Identifying Sections in Scientific Abstracts using Conditional Random Fields , 2008, IJCNLP.

[19]  Vincent Ng,et al.  Modeling Argument Strength in Student Essays , 2015, ACL.

[20]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[21]  Hwee Tou Ng,et al.  A PDTB-styled end-to-end discourse parser , 2012, Natural Language Engineering.

[22]  Man Lan,et al.  A Refined End-to-End Discourse Parser , 2015, CoNLL Shared Task.

[23]  Ani Nenkova,et al.  Automatic sense prediction for implicit discourse relations in text , 2009, ACL.

[24]  Phan Minh Dung,et al.  On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games , 1995, Artif. Intell..

[25]  Anthony Hunter,et al.  Elements of Argumentation , 2007, ECSQARU.

[26]  Günter Neumann,et al.  The Excitement Open Platform for Textual Inferences , 2014, ACL.

[27]  Claire Cardie,et al.  Identifying Appropriate Support for Propositions in Online User Comments , 2014, ArgMining@ACL.

[28]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[29]  Dietrich Rebholz-Schuhmann,et al.  Automatic recognition of conceptualization zones in scientific articles and two life science applications , 2012, Bioinform..

[30]  Jan Snajder,et al.  Back up your Stance: Recognizing Arguments in Online Discussions , 2014, ArgMining@ACL.

[31]  Kevin D. Ashley,et al.  Identifying Thesis and Conclusion Statements in Student Essays to Scaffold Peer Review , 2014, Intelligent Tutoring Systems.

[32]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[33]  Henry Prakken,et al.  Introduction to structured argumentation , 2014, Argument Comput..

[34]  Benno Stein,et al.  Using Argument Mining to Assess the Argumentation Quality of Essays , 2016, COLING.

[35]  Yue Zhang,et al.  Automatic Features for Essay Scoring – An Empirical Study , 2016, EMNLP.

[36]  David M. Blei,et al.  Probabilistic topic models , 2012, Commun. ACM.

[37]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[38]  Diane J. Litman,et al.  Ontology-Based Argument Mining and Automatic Essay Scoring , 2014, ArgMining@ACL.

[39]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[40]  Noam Slonim,et al.  Context Dependent Claim Detection , 2014, COLING.

[41]  Debanjan Ghosh,et al.  Coarse-grained Argumentation Features for Scoring Persuasive Essays , 2016, ACL.

[42]  Beata Beigman Klebanov,et al.  Argumentation: Content, Structure, and Relationship with Essay Quality , 2016, ArgMining@ACL.

[43]  Diane J. Litman,et al.  Extracting Argument and Domain Words for Identifying Argument Components in Texts , 2015, ArgMining@HLT-NAACL.

[44]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[45]  Jian Cheng,et al.  Validating automated speaking tests , 2010 .

[46]  Hwee Tou Ng,et al.  The CoNLL-2015 Shared Task on Shallow Discourse Parsing , 2015, CoNLL.

[47]  Roberto Navigli,et al.  Word sense disambiguation: A survey , 2009, CSUR.

[48]  Diane J. Litman,et al.  Improving Argument Mining in Student Essays by Learning and Exploiting Argument Indicators versus Essay Topics , 2016, FLAIRS Conference.

[49]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[50]  Semire Dikli,et al.  An Overview of Automated Scoring of Essays. , 2006 .

[51]  Andreas Peldszus,et al.  Towards segment-based recognition of argumentation structure in short texts , 2014, ArgMining@ACL.

[52]  Simone Teufel Towards Discipline-Independent Argumentative Zoning : Evidence from Chemistry and Computational Linguistics , 2009 .

[53]  Regina Barzilay,et al.  Bayesian Unsupervised Topic Segmentation , 2008, EMNLP.

[54]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[55]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[56]  Paolo Torroni,et al.  Argument Mining: A Machine Learning Perspective , 2015, TAFA.

[57]  Yulan He,et al.  Joint sentiment/topic model for sentiment analysis , 2009, CIKM.

[58]  Thomas Hofmann,et al.  Unsupervised Learning by Probabilistic Latent Semantic Analysis , 2004, Machine Learning.

[59]  Daniel Marcu,et al.  Finding the WRITE Stuff: Automatic Identification of Discourse Structure in Student Essays , 2003, IEEE Intell. Syst..

[60]  Anastassia Loukina,et al.  Feature selection for automated speech scoring , 2015, BEA@NAACL-HLT.

[61]  Dragomir R. Radev,et al.  Identifying Non-Explicit Citing Sentences for Citation-Based Summarization. , 2010, ACL.

[62]  Hwee Tou Ng,et al.  A Neural Approach to Automated Essay Scoring , 2016, EMNLP.

[63]  Xu Ling,et al.  Topic sentiment mixture: modeling facets and opinions in weblogs , 2007, WWW '07.

[64]  Bernard Moulin,et al.  A taxonomy of argumentation models used for knowledge representation , 2010, Artificial Intelligence Review.

[65]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[66]  Rob Grootendorst,et al.  The Speech Acts of Arguing and Convincing in Externalized Discussions , 1982, Reasonableness and Effectiveness in Argumentative Discourse.

[67]  Thomas Hofmann,et al.  Probabilistic Latent Semantic Analysis , 1999, UAI.

[68]  Sampo Pyysalo,et al.  Intrinsic Evaluation of Word Vectors Fails to Predict Extrinsic Performance , 2016, RepEval@ACL.

[69]  Xiaojin Zhu,et al.  A Topic Model for Word Sense Disambiguation , 2007, EMNLP.

[70]  Padhraic Smyth,et al.  Analyzing Entities and Topics in News Articles Using Statistical Topic Models , 2006, ISI.

[71]  Yee Whye Teh,et al.  Sharing Clusters among Related Groups: Hierarchical Dirichlet Processes , 2004, NIPS.

[72]  Yoichi Tomiura,et al.  Extracting Representative Words of a Topic Determined by Latent Dirichlet Allocation , 2014 .

[73]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[74]  Martin Chodorow,et al.  TOEFL11: A CORPUS OF NON‐NATIVE ENGLISH , 2013 .

[75]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[76]  Lucy Vanderwende,et al.  Exploring Content Models for Multi-Document Summarization , 2009, NAACL.

[77]  Marc Moens,et al.  Articles Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status , 2002, CL.

[78]  Thomas L. Griffiths,et al.  Integrating Topics and Syntax , 2004, NIPS.

[79]  Maria Liakata,et al.  Identifying the Information Structure of Scientific Abstracts: An Investigation of Three Different Schemes , 2010, BioNLP@ACL.

[80]  Alan Sergeant,et al.  Automatic Argumentation Extraction , 2013, ESWC.

[81]  Vasile Rus,et al.  SEMILAR: The Semantic Similarity Toolkit , 2013, ACL.

[82]  Christian D. Schunn,et al.  Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives , 2006 .

[83]  Iryna Gurevych,et al.  Identifying Argumentative Discourse Structures in Persuasive Essays , 2014, EMNLP.

[84]  Trevor J. M. Bench-Capon,et al.  Argumentation in artificial intelligence , 2007, Artif. Intell..

[85]  A. Knott,et al.  Using Linguistic Phenomena to Motivate a Set of Coherence Relations. , 1994 .

[86]  George E. Newell,et al.  Teaching and Learning Argumentative Reading and Writing: A Review of Research , 2011, Reading Research Quarterly.

[87]  Chen Wang,et al.  Introducing LUIMA: an experiment in legal conceptual retrieval of vaccine injury decisions using a UIMA type system and tools , 2015, ICAIL.

[88]  Kevin D. Ashley,et al.  Document Ranking with Citation Information and Oversampling Sentence Classification in the LUIMA Framework , 2016, JURIX.

[89]  Marie-Francine Moens,et al.  Automatic detection of arguments in legal texts , 2007, ICAIL.

[90]  Vangelis Karkaletsis,et al.  Argument Extraction from News , 2015, ArgMining@HLT-NAACL.

[91]  David M. Blei,et al.  Supervised Topic Models , 2007, NIPS.

[92]  Jacob Eisenstein,et al.  Representation Learning for Text-level Discourse Parsing , 2014, ACL.

[93]  Jan Snajder,et al.  TakeLab: Systems for Measuring Semantic Text Similarity , 2012, *SEMEVAL.

[94]  T. Govier A practical study of argument , 1985 .

[95]  David M. Williamson,et al.  A Framework for Evaluation and Use of Automated Scoring , 2012 .

[96]  Alice H. Oh,et al.  Aspect and sentiment unification model for online review analysis , 2011, WSDM '11.

[97]  Vincent Ng,et al.  End-to-End Argumentation Mining in Student Essays , 2016, NAACL.

[98]  Austin J. Freeley Argumentation and debate , 2010 .

[99]  Vincent Ng,et al.  Modeling Thesis Clarity in Student Essays , 2013, ACL.

[100]  Hongfei Yan,et al.  Jointly Modeling Aspects and Opinions with a MaxEnt-LDA Hybrid , 2010, EMNLP.

[101]  Fabio Ciravegna,et al.  Unsupervised document zone identification using probabilistic graphical models , 2012, LREC.

[102]  Serena Villata,et al.  From Discourse Analysis to Argumentation Schemes and Back: Relations and Differences , 2013, CLIMA.

[103]  Simone Teufel,et al.  Towards Domain-Independent Argumentative Zoning: Evidence from Chemistry and Computational Linguistics , 2009, EMNLP.

[104]  Christian D. Schunn,et al.  Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system , 2007, Comput. Educ..

[105]  James B. Freeman,et al.  Dialectics and the Macrostructure of Arguments , 1991 .

[106]  Serena Villata,et al.  Combining Textual Entailment and Argumentation Theory for Supporting Online Debates Interactions , 2012, ACL.

[107]  Vangelis Karkaletsis,et al.  Argument Extraction from News, Blogs, and Social Media , 2014, SETN.

[108]  Simone Teufel,et al.  Unsupervised learning of rhetorical structure with un-topic models , 2014, COLING.

[109]  Manfred Stede,et al.  Joint prediction in MST-style discourse parsing for argumentation mining , 2015, EMNLP.

[110]  Swapna Somasundaran,et al.  Recognizing Stances in Online Debates , 2009, ACL.

[111]  Nitin Madnani,et al.  Identifying High-Level Organizational Elements in Argumentative Discourse , 2012, NAACL.

[112]  Chris Reed,et al.  Argumentation Schemes , 2008 .

[113]  Iryna Gurevych,et al.  Parsing Argumentation Structures in Persuasive Essays , 2016, CL.

[114]  Manfred Stede,et al.  From Argument Diagrams to Argumentation Mining in Texts: A Survey , 2013, Int. J. Cogn. Informatics Nat. Intell..

[115]  David M. Williamson,et al.  Automated essay scoring: Psychometric guidelines and practices , 2013 .

[116]  Kevin D. Ashley,et al.  Improving Science Writing in Research Methods Classes Through Computerized Argument Diagramming , 2015, CogSci.

[117]  Noémie Elhadad,et al.  An Unsupervised Aspect-Sentiment Model for Online Reviews , 2010, NAACL.

[118]  Beata Beigman Klebanov,et al.  Applying Argumentation Schemes for Essay Scoring , 2014, ArgMining@ACL.

[119]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[120]  Diane J. Litman,et al.  Context-aware Argumentative Relation Mining , 2016, ACL.

[121]  Iryna Gurevych,et al.  Annotating Argument Components and Relations in Persuasive Essays , 2014, COLING.

[122]  Jill Burstein,et al.  Handbook of Automated Essay Evaluation Current Applications and New Directions , 2018 .

[123]  Karin Baier,et al.  The Uses Of Argument , 2016 .

[124]  Asher Stern,et al.  Design and realization of a modular architecture for textual entailment , 2013, Natural Language Engineering.

[125]  Ani Nenkova,et al.  What Makes Writing Great? First Experiments on Article Quality Prediction in the Science Journalism Domain , 2013, TACL.

[126]  Diane J. Litman,et al.  Automatic Scoring of an Analytical Response-To-Text Assessment , 2014, Intelligent Tutoring Systems.