Stance Classification in Rumours as a Sequential Task Exploiting the Tree Structure of Social Media Conversations

Rumour stance classification, the task that determines if each tweet in a collection discussing a rumour is supporting, denying, questioning or simply commenting on the rumour, has been attracting substantial interest. Here we introduce a novel approach that makes use of the sequence of transitions observed in tree-structured conversation threads in Twitter. The conversation threads are formed by harvesting users' replies to one another, which results in a nested tree-like structure. Previous work addressing the stance classification task has treated each tweet as a separate unit. Here we analyse tweets by virtue of their position in a sequence and test two sequential classifiers, Linear-Chain CRF and Tree CRF, each of which makes different assumptions about the conversational structure. We experiment with eight Twitter datasets, collected during breaking news, and show that exploiting the sequential structure of Twitter conversations achieves significant improvements over the non-sequential methods. Our work is the first to model Twitter conversations as a tree structure in this manner, introducing a novel way of tackling NLP tasks on Twitter conversations.

[1]  Arkaitz Zubiaga,et al.  Crowdsourcing the Annotation of Rumourous Conversations in Social Media , 2015, WWW.

[2]  Kalina Bontcheva,et al.  TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text , 2013, RANLP.

[3]  Kalina Bontcheva,et al.  Classifying Tweet Level Judgements of Rumours in Social Media , 2015, EMNLP.

[4]  Alan Ritter,et al.  Unsupervised Modeling of Twitter Conversations , 2010, NAACL.

[5]  M. Walker,et al.  How can you say such things?!?: Recognizing Disagreement in Informal Political Argument , 2011 .

[6]  Arkaitz Zubiaga,et al.  Microblog Analysis as a Program of Work , 2018, ACM Trans. Soc. Comput..

[7]  Xiaomo Liu,et al.  Real-time Rumor Debunking on Twitter , 2015, CIKM.

[8]  R. Procter,et al.  Reading the riots: what were the police doing on Twitter? , 2013 .

[9]  Kristin Precoda,et al.  Detection of Agreement and Disagreement in Broadcast Conversations , 2011, ACL.

[10]  Arkaitz Zubiaga,et al.  Microblog Analysis as a Programme of Work , 2015, ArXiv.

[11]  Qiaozhu Mei,et al.  Enquiring Minds: Early Detection of Rumors in Social Media from Enquiry Posts , 2015, WWW.

[12]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[13]  Rizal Setya Perdana What is Twitter , 2013 .

[14]  Shafiq R. Joty,et al.  Exploiting Conversational Features to Detect High-Quality Blog Comments , 2011, Canadian Conference on AI.

[15]  Li Zeng,et al.  #Unconfirmed: Classifying Rumor Stance in Crisis-Related Social Media Messages , 2016, ICWSM.

[16]  Andrew McCallum,et al.  An Introduction to Conditional Random Fields , 2010, Found. Trends Mach. Learn..

[17]  Mona T. Diab,et al.  Rumor Identification and Belief Investigation on Twitter , 2016, WASSA@NAACL-HLT.

[18]  R. Procter,et al.  Reading the riots on Twitter: methodological innovation for the analysis of big data , 2013 .

[19]  Hosung Park,et al.  What is Twitter, a social network or a news media? , 2010, WWW '10.

[20]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[21]  Dragomir R. Radev,et al.  Rumor has it: Identifying Misinformation in Microblogs , 2011, EMNLP.

[22]  Yang Liu,et al.  Finding Problem Solving Threads in Online Forum , 2011, IJCNLP.

[23]  Kathy McKeown,et al.  I Couldn't Agree More: The Role of Conversational Structure in Agreement and Disagreement Detection in Online Discussions , 2015, SIGDIAL Conference.

[24]  Saif Mohammad,et al.  SemEval-2016 Task 6: Detecting Stance in Tweets , 2016, *SEMEVAL.

[25]  Barbara Poblete,et al.  Twitter under crisis: can we trust what we RT? , 2010, SOMA '10.

[26]  Arkaitz Zubiaga,et al.  Hawkes Processes for Continuous Time Sequence Classification: an Application to Rumour Stance Classification in Twitter , 2016, ACL.

[27]  Preslav Nakov,et al.  SemEval-2015 Task 3: Answer Selection in Community Question Answering , 2015, *SEMEVAL.

[28]  Sven Behnke,et al.  PyStruct: learning structured prediction in python , 2014, J. Mach. Learn. Res..

[29]  Arkaitz Zubiaga,et al.  Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads , 2015, PloS one.