Stance Classification in Out-of-Domain Rumours: A Case Study Around Mental Health Disorders

Social media being a prolific source of rumours, stance classification of individual posts towards rumours has gained attention in the past few years. Classification of stance in individual posts can then be useful to determine the veracity of a rumour. Research in this direction has looked at rumours in different domains, such as politics, natural disasters or terrorist attacks. However, work has been limited to in-domain experiments, i.e. training and testing data belong to the same domain. This presents the caveat that when one wants to deal with rumours in domains that are more obscure, training data tends to be scarce. This is the case of mental health disorders, which we explore here. Having annotated collections of tweets around rumours emerged in the context of breaking news, we study the performance stability when switching to the new domain of mental health disorders. Our study confirms that performance drops when we apply our trained model on a new domain, emphasising the differences in rumours across domains. We overcome this issue by using a little portion of the target domain data for training, which leads to a substantial boost in performance. We also release the new dataset with mental health rumours annotated for stance.

[1]  Arkaitz Zubiaga,et al.  Detection and Resolution of Rumours in Social Media , 2017, ACM Comput. Surv..

[2]  Mona T. Diab,et al.  Rumor Detection and Classification for Twitter Data , 2015, ArXiv.

[3]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[4]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[5]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[6]  Georgiana Dinu,et al.  Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors , 2014, ACL.

[7]  Mona T. Diab,et al.  Rumor Identification and Belief Investigation on Twitter , 2016, WASSA@NAACL-HLT.

[8]  Weiwei Guo,et al.  Modeling Sentences in the Latent Space , 2012, ACL.

[9]  Li Zeng,et al.  #Unconfirmed: Classifying Rumor Stance in Crisis-Related Social Media Messages , 2016, ICWSM.

[10]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[11]  Arkaitz Zubiaga,et al.  Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads , 2015, PloS one.

[12]  Paul M. Thompson,et al.  Head Motion and Inattention/Hyperactivity Share Common Genetic Influences: Implications for fMRI Studies of ADHD , 2016, PloS one.

[13]  Percy Liang,et al.  Semi-Supervised Learning for Natural Language , 2005 .

[14]  Xiaomo Liu,et al.  Real-time Rumor Debunking on Twitter , 2015, CIKM.

[15]  Dragomir R. Radev,et al.  Rumor has it: Identifying Misinformation in Microblogs , 2011, EMNLP.

[16]  R. Stewart,et al.  Novel psychoactive substances: An investigation of temporal trends in social media and electronic health records , 2016, European Psychiatry.

[17]  Arkaitz Zubiaga,et al.  Stance Classification in Rumours as a Sequential Task Exploiting the Tree Structure of Social Media Conversations , 2016, COLING.

[18]  Kalina Bontcheva,et al.  Classifying Tweet Level Judgements of Rumours in Social Media , 2015, EMNLP.

[19]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[20]  Arindam Ghosh,et al.  In the mood for sharing contents: Emotions, personality and interaction styles in the diffusion of news , 2016, Inf. Process. Manag..

[21]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[22]  Arkaitz Zubiaga,et al.  Hawkes Processes for Continuous Time Sequence Classification: an Application to Rumour Stance Classification in Twitter , 2016, ACL.

[23]  Justin W. Patchin,et al.  Bullying, Cyberbullying, and Suicide , 2010, Archives of suicide research : official journal of the International Academy for Suicide Research.

[24]  Barbara Poblete,et al.  Twitter under crisis: can we trust what we RT? , 2010, SOMA '10.