IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks

Traditionally, a debate usually requires a manual preparation process, including reading plenty of articles, selecting the claims, identifying the stances of the claims, seeking the evidence for the claims, etc. As the AI debate attracts more attention these years, it is worth exploring the methods to automate the tedious process involved in the debating system. In this work, we introduce a comprehensive and large dataset named IAM, which can be applied to a series of argument mining tasks, including claim extraction, stance classification, evidence extraction, etc. Our dataset is collected from over 1k articles related to 123 topics. Near 70k sentences in the dataset are fully annotated based on their argument properties (e.g., claims, stances, evidence, etc.). We further propose two new integrated argument mining tasks associated with the debate preparation process: (1) claim extraction with stance classification (CESC) and (2) claim-evidence pair extraction (CEPE). We adopt a pipeline approach and an end-to-end method for each integrated task separately. Promising experimental results are reported to show the values and challenges of our proposed tasks, and motivate future research on argument mining.

[1]  Roy Bar-Haim,et al.  Advances in Debating Technologies: Building AI That Can Debate Humans , 2021, ACL.

[2]  Roy Bar-Haim,et al.  An autonomous debating system , 2021, Nature.

[3]  Qi Zhang,et al.  Discrete Argument Representation Learning for Interactive Argument Pair Identification , 2019, NAACL.

[4]  Lidong Bing,et al.  Argument Pair Extraction via Attention-guided Multi-Layer Multi-Cross Encoding , 2021, ACL.

[5]  Lidong Bing,et al.  Overview of Argumentative Text Understanding for AI Debater Challenge , 2021, NLPCC.

[6]  Qi Zhang,et al.  Leveraging Argumentation Knowledge Graph for Interactive Argument Pair Identification , 2021, FINDINGS.

[7]  Luo Si,et al.  APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning , 2020, EMNLP.

[8]  Claire Cardie,et al.  Exploring the Role of Argument Structure in Online Debate Persuasion , 2020, EMNLP.

[9]  Noam Slonim,et al.  A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis , 2019, AAAI.

[10]  Smaranda Muresan,et al.  AMPERSAND: Argument Mining for PERSuAsive oNline Discussions , 2019, EMNLP.

[11]  Michal Jacovi,et al.  Automatic Argument Quality Assessment - New Datasets and Methods , 2019, EMNLP.

[12]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[13]  Noam Slonim,et al.  Argument Invention from First Principles , 2019, ACL.

[14]  Chris Callison-Burch,et al.  Seeing Things from a Different Angle:Discovering Diverse Perspectives about Claims , 2019, NAACL.

[15]  Iryna Gurevych,et al.  Classification and Clustering of Arguments with Contextualized Word Embeddings , 2019, ACL.

[16]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[17]  Serena Villata,et al.  Increasing Argument Annotation Reproducibility by Using Inter-annotator Agreement to Improve Guidelines , 2018, LREC.

[18]  Serena Villata,et al.  Never Retreat, Never Retract: Argumentation Analysis for Political Speeches , 2018, AAAI.

[19]  Shachar Mirkin,et al.  Listening Comprehension over Argumentative Content , 2018, EMNLP.

[20]  Iryna Gurevych,et al.  What is the Essence of a Claim? Cross-Domain Claim Identification , 2017, EMNLP.

[21]  Iryna Gurevych,et al.  Neural End-to-End Learning for Computational Argumentation Mining , 2017, ACL.

[22]  Indrajit Bhattacharya,et al.  Stance Classification of Context-Dependent Claims , 2017, EACL.

[23]  Benno Stein,et al.  Computational Argumentation Quality Assessment in Natural Language , 2017, EACL.

[24]  Iryna Gurevych,et al.  Parsing Argumentation Structures in Persuasive Essays , 2016, CL.

[25]  Jan Snajder,et al.  Fill the Gap! Analyzing Implicit Premises between Claims from Online Debates , 2016, ArgMining@ACL.

[26]  Iryna Gurevych,et al.  Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidirectional LSTM , 2016, ACL.

[27]  Vincent Ng,et al.  End-to-End Argumentation Mining in Student Essays , 2016, NAACL.

[28]  Paolo Torroni,et al.  Argument Mining from Speech: Detecting Claims in Political Debates , 2016, AAAI.

[29]  Chris Reed,et al.  Mining Ethos in Political Debate , 2016, COMMA.

[30]  Mitesh M. Khapra,et al.  Show Me Your Evidence - an Automatic Method for Context Dependent Evidence Detection , 2015, EMNLP.

[31]  Chen Wang,et al.  Introducing LUIMA: an experiment in legal conceptual retrieval of vaccine injury decisions using a UIMA type system and tools , 2015, ICAIL.

[32]  Iryna Gurevych,et al.  Identifying Argumentative Discourse Structures in Persuasive Essays , 2014, EMNLP.

[33]  Noam Slonim,et al.  Context Dependent Claim Detection , 2014, COLING.

[34]  Noam Slonim,et al.  A Benchmark Dataset for Automatic Detection of Claims and Evidence in the Context of Controversial Topics , 2014, ArgMining@ACL.

[35]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[36]  Serena Villata,et al.  Combining Textual Entailment and Argumentation Theory for Supporting Online Debates Interactions , 2012, ACL.

[37]  Marie-Francine Moens,et al.  Argumentation mining , 2011, Artificial Intelligence and Law.

[38]  Wiebke Wagner,et al.  Steven Bird, Ewan Klein and Edward Loper: Natural Language Processing with Python, Analyzing Text with the Natural Language Toolkit , 2010, Lang. Resour. Evaluation.