Representation and learning schemes for argument stance mining

Argumentation is a key part of human interaction. Used introspectively, it searches for the truth, by laying down argument for and against positions. As a mediation tool, it can be used to search for compromise between multiple human agents. For this purpose, theories of argumentation have been in development since the Ancient Greeks in order to formalise the process and therefore remove the human imprecision from it. From this practice the process of argument mining has emerged. As human interaction has moved from the small scale of one-to-one (or few-to-few) debates to large scale discussions where tens of thousands of participants can express their opinion in real time, the importance of argument mining has grown while its feasibility in a manual annotation setting has diminished and relied mainly on a human-defined heuristics to process the data. This underlines the importance of a new generation of computational tools that can automate this process on a larger scale. In this thesis we study argument stance detection, one of the steps involved in the argument mining workflow. We demonstrate how we can use data of varying reliability in order to mine argument stance in social media data. We investigate a spectrum of techniques, from completely unsupervised classification of stance using a sentiment lexicon, automated computation of a regularised stance lexicon, automated computation of a lexicon with modifiers, and the use of a lexicon with modifiers as a temporal feature model for more complex classification algorithms. We find that the addition of contextual information enhances unsupervised stance classification, within reason, and that multi-strategy algorithms that combine multiple heuristics by ordering them from the precise to the general tend to outperform other approaches by a large margin. Focusing then on building a stance lexicon, we find that optimising such lexicons using an empirical risk minimisation framework allows us to regularise them to a higher degree than competing probabilistic techniques, which helps us learn better lexicons from noisy data. We also conclude that adding local context (neighbouring words) information during the learning phase of the lexicons tends to produce more accurate results at the cost of robustness, since part of the weights is distributed from the words with a class valence to the contextual words. Finally, when investigating the use of lexicons to build feature models for traditional machine learning techniques, simple lexicons (without context) seem to perform overall as well as more complex ones, and better than purely semantic representations. We also find that word-level feature models tend to outperform sentence and instance-level representations, but that they do not benefit as much from being augmented by lexicon knowledge. This research programme was carried out in collaboration with the University of Glasgow, Department of Computer Science.

[1]  Theresa Wilson,et al.  Agreement detection in multiparty conversation , 2009, ICMI-MLMI '09.

[2]  Yu Zhou,et al.  Overview of NLPCC Shared Task 4: Stance Detection in Chinese Microblogs , 2016, NLPCC/ICCPOL.

[3]  Stan Matwin,et al.  From Argumentation Mining to Stance Classification , 2015, ArgMining@HLT-NAACL.

[4]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[5]  Enhong Chen,et al.  Word Embedding Revisited: A New Representation Learning and Explicit Matrix Factorization Perspective , 2015, IJCAI.

[6]  Andrea Esuli,et al.  SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining , 2006, LREC.

[7]  Marie-Francine Moens,et al.  Automatic detection of arguments in legal texts , 2007, ICAIL.

[8]  Bart Verheij,et al.  On the existence and multiplicity of extensions in dialectical argumentation , 2002, NMR.

[9]  Brendan T. O'Connor,et al.  Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks , 2008, EMNLP.

[10]  Phan Minh Dung,et al.  On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games , 1995, Artif. Intell..

[11]  M. Walker,et al.  How can you say such things?!?: Recognizing Disagreement in Informal Political Argument , 2011 .

[12]  Chris Reed,et al.  Araucaria: Software for Argument Analysis, Diagramming and Representation , 2004, Int. J. Artif. Intell. Tools.

[13]  Mona T. Diab,et al.  CU-GWU Perspective at SemEval-2016 Task 6: Ideological Stance Detection in Informal Text , 2016, *SEMEVAL.

[14]  Michael I. Jordan Serial Order: A Parallel Distributed Processing Approach , 1997 .

[15]  Jean Carletta,et al.  The AMI meeting corpus , 2005 .

[16]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[17]  Amita Misra,et al.  Topic Independent Identification of Agreement and Disagreement in Social Media Dialogue , 2013, SIGDIAL Conference.

[18]  Anthony Hunter,et al.  Elements of Argumentation , 2007, ECSQARU.

[19]  Michael Rabadi,et al.  Kernel Methods for Machine Learning , 2015 .

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Ellen Riloff,et al.  Learning Extraction Patterns for Subjective Expressions , 2003, EMNLP.

[22]  Jan Snajder,et al.  Back up your Stance: Recognizing Arguments in Online Discussions , 2014, ArgMining@ACL.

[23]  Tommi S. Jaakkola,et al.  Maximum Entropy Discrimination , 1999, NIPS.

[24]  Y. Nesterov Gradient methods for minimizing composite objective function , 2007 .

[25]  Tejashri Inadarchand Jain,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2010 .

[26]  Ronan Collobert,et al.  Word Embeddings through Hellinger PCA , 2013, EACL.

[27]  Adam Faulkner,et al.  Automated Classification of Stance in Student Essays: An Approach Using Stance Target Information and the Wikipedia Link-Based Measure , 2014, FLAIRS.

[28]  Richard E. Ladner,et al.  Agreement/Disagreement Classification: Exploiting Unlabeled Data using Contrast Classifiers , 2006, HLT-NAACL.

[29]  Marilyn A. Walker,et al.  A Corpus for Research on Deliberation and Debate , 2012, LREC.

[30]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[31]  Patrick Paroubek,et al.  Twitter as a Corpus for Sentiment Analysis and Opinion Mining , 2010, LREC.

[32]  Andreas Stolcke,et al.  The ICSI Meeting Corpus , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[33]  Nan Yu,et al.  Stance Detection in Chinese MicroBlogs with Neural Networks , 2016, NLPCC/ICCPOL.

[34]  Enric Plaza,et al.  Sentiment and Preference Guided Social Recommendation , 2014, ICCBR.

[35]  James R. Foulds,et al.  Joint Models of Disagreement and Stance in Online Debate , 2015, ACL.

[36]  R. Gordon Shallow techniques for argument mining , 2017 .

[37]  Ruifeng Xu,et al.  Stance Classification with Target-specific Neural Attention , 2017, IJCAI.

[38]  Marilyn A. Walker,et al.  Cats Rule and Dogs Drool!: Classifying Stance in Online Debate , 2011, WASSA@ACL.

[39]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[40]  Annie Zaenen,et al.  Contextual Valence Shifters , 2006, Computing Attitude and Affect in Text.

[41]  Nirmalie Wiratunga,et al.  A Hybrid Sentiment Lexicon for Social Media Mining , 2014, 2014 IEEE 26th International Conference on Tools with Artificial Intelligence.

[42]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[43]  Braja Gopal Patra,et al.  JU_NLP at SemEval-2016 Task 6: Detecting Stance in Tweets using Support Vector Machines , 2016, *SEMEVAL.

[44]  Marshall S. Smith,et al.  The general inquirer: A computer approach to content analysis. , 1967 .

[45]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Загоровская Ольга Владимировна,et al.  Исследование влияния пола и психологических характеристик автора на количественные параметры его текста с использованием программы Linguistic Inquiry and Word Count , 2015 .

[47]  I. Hutchby Confrontation Talk: Arguments, Asymmetries, and Power on Talk Radio , 1996 .

[48]  Marie-Francine Moens,et al.  Argumentation mining: the detection, classification and structure of arguments in text , 2009, ICAIL.

[49]  Vasileios Hatzivassiloglou,et al.  Predicting the Semantic Orientation of Adjectives , 1997, ACL.

[50]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[51]  Lianhong Cai,et al.  Recognizing stances in Mandarin social ideological debates with text and acoustic features , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[52]  Karin Becker,et al.  INF-UFRGS-OPINION-MINING at SemEval-2016 Task 6: Automatic Generation of a Training Corpus for Unsupervised Identification of Stance in Tweets , 2016, *SEMEVAL.

[53]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[54]  Nirmalie Wiratunga,et al.  Contextual sentiment analysis for social media genres , 2016, Knowl. Based Syst..

[55]  Marilyn A. Walker,et al.  That is your evidence?: Classifying stance in online political debate , 2012, Decis. Support Syst..

[56]  Yong Yu,et al.  “We make choices we think are going to save us”: Debate and stance identification for online breast cancer CAM discussions , 2017, WWW.

[57]  Naoaki Okazaki,et al.  Tohoku at SemEval-2016 Task 6: Feature-based Model versus Convolutional Neural Network for Stance Detection , 2016, *SEMEVAL.

[58]  N. Wiratunga,et al.  Towards Argumentative Opinion Mining in Online Discussions , 2014 .

[59]  Floris Bex,et al.  Implementing the argument web , 2013, Commun. ACM.

[60]  Claire Cardie,et al.  Improving Agreement and Disagreement Identification in Online Discussions with A Socially-Tuned Sentiment Lexicon , 2014, WASSA@ACL.

[61]  Claudette Cayrol,et al.  On bipolarity in argumentation frameworks , 2008, NMR.

[62]  Kathy McKeown,et al.  I Couldn't Agree More: The Role of Conversational Structure in Agreement and Disagreement Detection in Online Discussions , 2015, SIGDIAL Conference.

[63]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[64]  Subrata Ghosh,et al.  Unsupervised stance classification in online debates , 2018, COMAD/CODS.

[65]  Guido Zarrella,et al.  MITRE at SemEval-2016 Task 6: Transfer Learning for Stance Detection , 2016, *SEMEVAL.

[66]  Tim Berners-Lee,et al.  Information Management: A Proposal , 1990 .

[67]  Stewart Massie,et al.  Generating a Word-Emotion Lexicon from #Emotional Tweets , 2014, *SEMEVAL.

[68]  C. Cayrol,et al.  On the Acceptability of Arguments in Bipolar Argumentation Frameworks , 2005, ECSQARU.

[69]  Martine De Cock,et al.  Fuzzy Argumentation Frameworks , 2008 .

[70]  Jacob Andreas,et al.  Annotating Agreement and Disagreement in Threaded Discussion , 2012, LREC.

[71]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[72]  Chris Reed,et al.  OVA+: an Argument Analysis Interface , 2014, COMMA.

[73]  Kristin Precoda,et al.  Detection of Agreement and Disagreement in Broadcast Conversations , 2011, ACL.

[74]  Nirmalie Wiratunga,et al.  Neural Induction of a Lexicon for Fast and Interpretable Stance Classification , 2017, LDK.

[75]  Guillaume Cabanac,et al.  Predicting Emotional Reaction in Social Networks , 2017, ECIR.

[76]  Zachary Chase Lipton The mythos of model interpretability , 2016, ACM Queue.

[77]  Torsten Zesch,et al.  ltl.uni-due at SemEval-2016 Task 6: Stance Detection in Social Media Using Stacked Classifiers , 2016, *SEMEVAL.

[78]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[79]  Guodong Zhou,et al.  Exploring Various Linguistic Features for Stance Detection , 2016, NLPCC/ICCPOL.

[80]  Kalina Bontcheva,et al.  USFD at SemEval-2016 Task 6: Any-Target Stance Detection on Twitter with Autoencoders , 2016, *SEMEVAL.

[81]  Soroush Vosoughi,et al.  DeepStance at SemEval-2016 Task 6: Detecting Stance in Tweets Using Character and Word-Level CNNs , 2016, *SEMEVAL.

[82]  Eric P. Xing,et al.  Staying Informed: Supervised and Semi-Supervised Multi-View Topical Analysis of Ideological Perspective , 2010, EMNLP.

[83]  Andreas Vlachos,et al.  Emergent: a novel data-set for stance classification , 2016, NAACL.

[84]  Vincent Ng,et al.  Stance Classification of Ideological Debates: Data, Models, Features, and Constraints , 2013, IJCNLP.

[85]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[86]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[87]  Amita Misra,et al.  NLDS-UCSC at SemEval-2016 Task 6: A Semi-Supervised Approach to Detecting Stance in Tweets , 2016, *SEMEVAL.

[88]  Stewart Massie,et al.  Lexicon based feature extraction for emotion text classification , 2017, Pattern Recognit. Lett..

[89]  Thomas Demeester,et al.  Representation learning for very short texts using weighted word embedding aggregation , 2016, Pattern Recognit. Lett..

[90]  Julia Hirschberg,et al.  Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies , 2004, ACL.

[91]  Saif Mohammad,et al.  Stance and Sentiment in Tweets , 2016, ACM Trans. Internet Techn..

[92]  Mari Ostendorf,et al.  Detection Of Agreement vs. Disagreement In Meetings: Training With Unlabeled Data , 2003, NAACL.

[93]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[94]  Josef Steinberger,et al.  UWB at SemEval-2016 Task 6: Stance Detection , 2016, *SEMEVAL.

[95]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[96]  Saif Mohammad,et al.  SemEval-2016 Task 6: Detecting Stance in Tweets , 2016, *SEMEVAL.

[97]  Eric Horvitz,et al.  Crowdsourcing the acquisition of natural language corpora: Methods and observations , 2012, 2012 IEEE Spoken Language Technology Workshop (SLT).

[98]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[99]  Nirmalie Wiratunga,et al.  Domain-Based Lexicon Enhancement for Sentiment Analysis , 2013, SMA@BCS-SGAI.

[100]  Swapna Somasundaran,et al.  Recognizing Stances in Ideological On-Line Debates , 2010, HLT-NAACL 2010.

[101]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[102]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[103]  Martin Tutek,et al.  TakeLab at SemEval-2016 Task 6: Stance Classification in Tweets Using a Genetic Algorithm Based Ensemble , 2016, *SEMEVAL.

[104]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[105]  Dragomir R. Radev,et al.  What’s with the Attitude? Identifying Sentences with Attitude in Online Discussions , 2010, EMNLP.

[106]  Rune Sætre,et al.  IDI$@$NTNU at SemEval-2016 Task 6: Detecting Stance in Tweets Using Shallow Features and GloVe Vectors for Word Representation , 2016, SemEval@NAACL-HLT.

[107]  Yue Chen,et al.  IUCL at SemEval-2016 Task 6: An Ensemble Model for Stance Detection in Twitter , 2016, *SEMEVAL.

[108]  Dragomir R. Radev,et al.  Detecting Subgroups in Online Discussions by Modeling Positive and Negative Relations among Participants , 2012, EMNLP.

[109]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[110]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[111]  Karin Baier,et al.  The Uses Of Argument , 2016 .

[112]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[113]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[114]  Torsten Zesch,et al.  Stance-based Argument Mining - Modeling Implicit Argumentation Using Stance , 2016, KONVENS.

[115]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[116]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[117]  Vincent Ng,et al.  Extra-Linguistic Constraints on Stance Recognition in Ideological Debates , 2013, ACL.

[118]  Kalina Bontcheva,et al.  Stance Detection with Bidirectional Conditional Encoding , 2016, EMNLP.

[119]  Marilyn A. Walker,et al.  Collective Stance Classification of Posts in Online Debate Forums , 2014 .

[120]  Zhihua Zhang,et al.  ECNU at SemEval 2016 Task 6: Relevant or Not? Supportive or Not? A Two-step Learning System for Automatic Detecting Stance in Tweets , 2016, SemEval@NAACL-HLT.

[121]  Peter D. Turney Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[122]  Omer Levy,et al.  Neural Word Embedding as Implicit Matrix Factorization , 2014, NIPS.

[123]  Venkatesh Duppada,et al.  Agree to disagree: Improving disagreement detection with dual GRUs , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW).

[124]  Igor Mozetič,et al.  Stance and influence of Twitter users regarding the Brexit referendum , 2017, Computational social networks.

[125]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[126]  Paul Thomas,et al.  Unifying Local and Global Agreement and Disagreement Classification in Online Debates , 2012, WASSA@ACL.

[127]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[128]  Gal Chechik,et al.  Euclidean Embedding of Co-occurrence Data , 2004, J. Mach. Learn. Res..

[129]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[130]  Marilyn A. Walker,et al.  Stance Classification using Dialogic Properties of Persuasion , 2012, NAACL.

[131]  Dimitris Papadias,et al.  Computer supported argumentation and collaborative decision making: the HERMES system , 2001, Inf. Syst..

[132]  Hatem A. Fayed,et al.  Stance Detection in Tweets Using a Majority Vote Classifier , 2018, AMLTA.

[133]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[134]  Arkaitz Zubiaga,et al.  Discourse-aware rumour stance classification in social media using sequential classifiers , 2017, Inf. Process. Manag..

[135]  Xiao Zhang,et al.  pkudblab at SemEval-2016 Task 6 : A Specific Convolutional Neural Network System for Effective Stance Detection , 2016, *SEMEVAL.