Automatic detection and verification of rumors on Twitter

The spread of malicious or accidental misinformation in social media, especially in timesensitive situations such as real-world emergencies can have harmful effects on individuals and society. This thesis develops models for automated detection and verification of rumors (unverified information) that propagate through Twitter. Detection of rumors about an event is achieved through classifying and clustering assertions made about that event. Assertions are classified through a speech-act classifier for Twitter developed for this thesis. The classifier utilizes a combination of semantic and syntactic features to identify assertions with 91% accuracy. To predict the veracity of rumors, we identify salient features of rumors by examining three aspects of information spread: linguistic style used to express rumors, characteristics of people involved in propagating information, and network propagation dynamics. The predicted veracity of a time series of these features extracted from a rumor (a collection of tweets) is generated using Hidden Markov Models. The verification algorithm was tested on 209 rumors representing 938,806 tweets collected from real-world events including the 2013 Boston Marathon bombings, the 2014 Ferguson unrest and the 2014 Ebola epidemic, and many other rumors reported on popular websites that document public rumors. The algorithm is able to predict the veracity of rumors with an accuracy of 75%. The ability to track rumors and predict their outcomes may have practical applications for news consumers, financial markets, journalists, and emergency services, and more generally to help minimize the impact of false information on Twitter. Thesis Supervisor: Deb K. Roy Title: Associate Professor, Program in Media Arts and Sciences

[1]  R. L. Rosnow,et al.  Rumor rest stops on the information highway: Transmission patterns in a computer-mediated rumor chain , 1998 .

[2]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[3]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[4]  Junlan Feng,et al.  Robust Sentiment Detection on Twitter from Biased and Noisy Data , 2010, COLING.

[5]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[6]  Noah A. Smith,et al.  A Dependency Parser for Tweets , 2014, EMNLP.

[7]  Stephen Dann,et al.  Twitter content classification , 2010, First Monday.

[8]  Sinan Aral,et al.  Identifying Influential and Susceptible Members of Social Networks , 2012, Science.

[9]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[10]  Alok N. Choudhary,et al.  Twitter Trending Topic Classification , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[11]  Jure Leskovec,et al.  Meme-tracking and the dynamics of the news cycle , 2009, KDD.

[12]  P. M. Greco When the tail wags the dog. , 2011, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[13]  Hosung Park,et al.  What is Twitter, a social network or a news media? , 2010, WWW '10.

[14]  Fan Yang,et al.  Automatic detection of rumor on Sina Weibo , 2012, MDS '12.

[15]  Leysia Palen,et al.  Microblogging during two natural hazards events: what twitter may contribute to situational awareness , 2010, CHI.

[16]  Kristian J. Hammond,et al.  Domain Specific Affective Classification of Documents , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[17]  P. Earle,et al.  OMG Earthquake! Can Twitter Improve Earthquake Response? , 2009 .

[18]  Alan F. Smeaton,et al.  Topic-dependent sentiment analysis of financial blogs , 2009, TSA@CIKM.

[19]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[20]  Jean Aitchison,et al.  Language and the Internet , 2002, Lit. Linguistic Comput..

[21]  Christos Faloutsos,et al.  Rise and fall patterns of information diffusion: model and implications , 2012, KDD.

[22]  Kentaro Inui,et al.  Dependency Tree-based Sentiment Classification using CRFs with Hidden Variables , 2010, NAACL.

[23]  Timothy W. Finin,et al.  Why we twitter: understanding microblogging usage and communities , 2007, WebKDD/SNA-KDD '07.

[24]  Ari Rappoport,et al.  Enhanced Sentiment Learning Using Twitter Hashtags and Smileys , 2010, COLING.

[25]  Soroush Vosoughi,et al.  Tweet Acts: A Speech Act Classifier for Twitter , 2016, ICWSM.

[26]  Elizabeth Shriberg,et al.  Meeting Recorder Project: Dialog Act Labeling Guide , 2004 .

[27]  Brendan T. O'Connor,et al.  Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters , 2013, NAACL.

[28]  David A. Shamma,et al.  Characterizing debate performance via aggregated twitter sentiment , 2010, CHI.

[29]  Timothy Baldwin,et al.  Automatic Satire Detection: Are You Having a Laugh? , 2009, ACL.

[30]  Leysia Palen,et al.  Twitter adoption and use in mass convergence and emergency events , 2009 .

[31]  Bertrand De Longueville,et al.  "OMG, from here, I can see the flames!": a use case of mining location based social networks to acquire spatio-temporal data on forest fires , 2009, LBSN '09.

[32]  J. Searle Expression and Meaning: Studies in the Theory of Speech Acts , 1979 .

[33]  R. Kreuz,et al.  Lexical Influences on the Perception of Sarcasm , 2007 .

[34]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[35]  T. Shibutani Improvised News: A Sociological Study of Rumor , 1966 .

[36]  John Yen,et al.  Finding influential users of online health communities: a new metric based on sentiment influence. , 2014, Journal of the American Medical Informatics Association : JAMIA.

[37]  Jacob Ratkiewicz,et al.  Detecting and Tracking the Spread of Astroturf Memes in Microblog Streams , 2010, ArXiv.

[38]  Wenjie Li,et al.  Towards Scalable Speech Act Recognition in Twitter: Tackling Insufficient Training Data , 2012 .

[39]  Elizabeth Shriberg,et al.  Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual , 1997 .

[40]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[41]  D. C. Kirkpatrick,et al.  Relating Difficulty : The Processes of Constructing and Managing Difficult Interaction , 2006 .

[42]  D. Watts,et al.  Influentials, Networks, and Public Opinion Formation , 2007 .

[43]  Rizal Setya Perdana What is Twitter , 2013 .

[44]  Patrick Paroubek,et al.  Twitter as a Corpus for Sentiment Analysis and Opinion Mining , 2010, LREC.

[45]  A. Wierzbicka English Speech Act Verbs: A Semantic Dictionary , 1987 .

[46]  Devavrat Shah,et al.  Rumors in a Network: Who's the Culprit? , 2009, IEEE Transactions on Information Theory.

[47]  Mor Naaman,et al.  Is it really about me?: message content in social awareness streams , 2010, CSCW '10.

[48]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[49]  Tiejun Zhao,et al.  Target-dependent Twitter Sentiment Classification , 2011, ACL.

[50]  Nello Cristianini,et al.  Flu Detector - Tracking Epidemics on Twitter , 2010, ECML/PKDD.

[51]  Din J. Wasem,et al.  Mining of Massive Datasets , 2014 .

[52]  Gerardo Iñiguez,et al.  Complex contagion process in spreading of online innovation , 2014, Journal of The Royal Society Interface.

[53]  Hanan Samet,et al.  TwitterStand: news in tweets , 2009, GIS.

[54]  Justin Cheng,et al.  Rumor Cascades , 2014, ICWSM.

[55]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[56]  Damon Centola,et al.  The Spread of Behavior in an Online Social Network Experiment , 2010, Science.

[57]  Wilma Stassen Your news in 140 characters: exploring the role of social media in journalism , 2011 .

[58]  Elizabeth Shriberg,et al.  Automatic dialog act segmentation and classification in multiparty meetings , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[59]  Kirill Kireyev Applications of Topics Models to Analysis of Disaster-Related Twitter Data , 2009 .

[60]  Xiaolong Wang,et al.  Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach , 2011, CIKM '11.

[61]  E. Steyerberg,et al.  [Regression modeling strategies]. , 2011, Revista espanola de cardiologia.

[62]  Leysia Palen,et al.  Chatter on the red: what hazards threat reveals about the social life of microblogged information , 2010, CSCW '10.

[63]  Jacob Ratkiewicz,et al.  Detecting and Tracking Political Abuse in Social Media , 2011, ICWSM.

[64]  Duncan J. Watts,et al.  Everyone's an influencer: quantifying influence on twitter , 2011, WSDM '11.

[65]  Julia Hirschberg,et al.  Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies , 2004, ACL.

[66]  Hugo Liu,et al.  ConceptNet — A Practical Commonsense Reasoning Tool-Kit , 2004 .

[67]  Jonathon Read,et al.  Using Emoticons to Reduce Dependency in Machine Learning Techniques for Sentiment Classification , 2005, ACL.

[68]  Hakan Ferhatosmanoglu,et al.  Short text classification in twitter to improve information filtering , 2010, SIGIR.

[69]  Johanna D. Moore,et al.  Twitter Sentiment Analysis: The Good the Bad and the OMG! , 2011, ICWSM.

[70]  Andreas Stolcke,et al.  Dialogue act modeling for automatic tagging and recognition of conversational speech , 2000, CL.

[71]  J. Pennebaker,et al.  Psychological aspects of natural language. use: our words, our selves. , 2003, Annual review of psychology.

[72]  Harith Alani,et al.  Alleviating Data Sparsity for Twitter Sentiment Analysis , 2012, #MSM.

[73]  Wenjie Li,et al.  What Are Tweeters Doing: Recognizing Speech Acts in Twitter , 2011, Analyzing Microtext.

[74]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[75]  Ewan Klein,et al.  Natural Language Processing with Python , 2009 .

[76]  Kyomin Jung,et al.  Prominent Features of Rumor Propagation in Online Social Media , 2013, 2013 IEEE 13th International Conference on Data Mining.

[77]  M. Csíkszentmihályi,et al.  Happiness in Everyday Life: The Uses of Experience Sampling , 2003 .

[78]  M. Newman Spread of epidemic disease on networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[79]  Soroush Vosoughi,et al.  Enhanced Twitter Sentiment Classification Using Contextual Information , 2015, WASSA@EMNLP.

[80]  Donald F. Towsley,et al.  The effect of network topology on the spread of epidemics , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[81]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[82]  Marshall S. Smith,et al.  The general inquirer: A computer approach to content analysis. , 1967 .

[83]  B. Efron The jackknife, the bootstrap, and other resampling plans , 1987 .

[84]  Alessandro Vespignani,et al.  Epidemic spreading in scale-free networks. , 2000, Physical review letters.

[85]  Johan Bollen,et al.  Modeling Public Mood and Emotion: Twitter Sentiment and Socio-Economic Phenomena , 2009, ICWSM.

[86]  Robin Cowan,et al.  Network Structure and the Diffusion of Knowledge , 2004 .

[87]  R. L. Rosnow Inside rumor: A personal journey. , 1991 .

[88]  Slava M. Katz,et al.  Estimation of probabilities from sparse data for the language model component of a speech recognizer , 1987, IEEE Trans. Acoust. Speech Signal Process..

[89]  Jing Jiang,et al.  An Empirical Comparison of Topics in Twitter and Traditional Media , 2011 .

[90]  Chang-Tien Lu,et al.  Misinformation Propagation in the Age of Twitter , 2014, Computer.

[91]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[92]  Max Kaufmann Syntactic Normalization of Twitter Messages , 2010 .

[93]  Barbara Poblete,et al.  Twitter under crisis: can we trust what we RT? , 2010, SOMA '10.

[94]  Ellen Spertus,et al.  Smokey: Automatic Recognition of Hostile Messages , 1997, AAAI/IAAI.

[95]  Daniel G. Goldstein,et al.  The structure of online diffusion networks , 2012, EC '12.

[96]  Hiroya Takamura,et al.  Sentiment Classification Using Word Sub-sequences and Dependency Sub-trees , 2005, PAKDD.

[97]  Barbara Poblete,et al.  Information credibility on twitter , 2011, WWW.

[98]  Stephen Wu,et al.  Well-Being across America , 2011, Review of Economics and Statistics.

[99]  Gary Geunbae Lee,et al.  Semi-supervised Speech Act Recognition in Emails and Forums , 2009, EMNLP.