Monitoring Misinformation on Twitter During Crisis Events: A Machine Learning Approach

Social media has been increasingly utilized to spread breaking news and risk communications during disasters of all magnitudes. Unfortunately, due to the unmoderated nature of social media platforms such as Twitter, rumors and misinformation are able to propagate widely. Given this, a surfeit of research has studied false rumor diffusion on Twitter, especially during natural disasters. Within this domain, studies have also focused on the misinformation control efforts from government organizations and other major agencies. A prodigious gap in research exists in studying the monitoring of misinformation on social media platforms in times of disasters and other crisis events. Such studies would offer organizations and agencies new tools and ideologies to monitor misinformation on platforms such as Twitter, and make informed decisions on whether or not to use their resources in order to debunk. In this work, we fill the research gap by developing a machine learning framework to predict the veracity of tweets that are spread during crisis events. The tweets are tracked based on the veracity of their content as either true, false, or neutral. We conduct four separate studies, and the results suggest that our framework is capable of tracking multiple cases of misinformation simultaneously, with F 1 scores exceeding 87%. In the case of tracking a single case of misinformation, our framework reaches an F 1 score of 83%. We collect and drive the algorithms with 15,952 misinformation-related tweets from the Boston Marathon bombing (2013), Manchester Arena bombing (2017), Hurricane Harvey (2017), Hurricane Irma (2017), and the Hawaii ballistic missile false alert (2018). This article provides novel insights on how to efficiently monitor misinformation that is spread during disasters.

[1]  Eui-Nam Huh,et al.  Harness Human Sensor Networks for Situational Awareness in Disaster Reliefs: A Survey , 2013 .

[2]  Patric R. Spence,et al.  Variability in Twitter Content Across the Stages of a Natural Disaster: Implications for Crisis Communication , 2015 .

[3]  Yuko Murayama,et al.  Trust Issues in Disaster Communications , 2013, 2013 46th Hawaii International Conference on System Sciences.

[4]  Y. Lin,et al.  Tracking and Analyzing Individual Distress Following Terrorist Attacks Using Social Media Streams , 2017, Risk analysis : an official publication of the Society for Risk Analysis.

[5]  Xiaofei Xie,et al.  The Role of Emotions in Risk Communication , 2011, Risk analysis : an official publication of the Society for Risk Analysis.

[6]  José Manuel Benítez,et al.  On the usefulness of cross-validation for directional forecast evaluation , 2014, Comput. Stat. Data Anal..

[7]  Kyomin Jung,et al.  Prominent Features of Rumor Propagation in Online Social Media , 2013, 2013 IEEE 13th International Conference on Data Mining.

[8]  Kevin Driscoll,et al.  Political rumoring on Twitter during the 2012 US presidential election: Rumor diffusion and correction , 2017, New Media Soc..

[9]  J. Zhuang,et al.  Crisis information distribution on Twitter: a content analysis of tweets during Hurricane Sandy , 2017, Natural Hazards.

[10]  Tomer Simon,et al.  Socializing in emergencies - A review of the use of social media in emergency situations , 2015, Int. J. Inf. Manag..

[11]  Barbara Poblete,et al.  Predicting information credibility in time-sensitive social media , 2013, Internet Res..

[12]  JinKyu Lee,et al.  Web 2.0 Emergency Applications: How Useful Can Twitter be for Emergency Response? , 2009 .

[13]  Yue Yu,et al.  Retweeting Risk Communication: The Role of Threat and Efficacy , 2018, Risk analysis : an official publication of the Society for Risk Analysis.

[14]  Jun Zhuang,et al.  An Analysis of Public Opinions Regarding Take-Away Food Safety: A 2015–2018 Case Study on Sina Weibo , 2020, Foods.

[15]  Arkaitz Zubiaga,et al.  Hawkes Processes for Continuous Time Sequence Classification: an Application to Rumour Stance Classification in Twitter , 2016, ACL.

[16]  Arkaitz Zubiaga,et al.  Analysing How People Orient to and Spread Rumours in Social Media by Looking at Conversational Threads , 2015, PloS one.

[17]  Bairong Wang,et al.  Rumor response, debunking response, and decision makings of misinformed Twitter users during disasters , 2018, Natural Hazards.

[18]  Heather Lazrus,et al.  Factors Affecting Hurricane Evacuation Intentions , 2015, Risk analysis : an official publication of the Society for Risk Analysis.

[19]  Haimonti Dutta,et al.  A system for intergroup prejudice detection: The case of microblogging under terrorist attacks , 2018, Decis. Support Syst..

[20]  Baruch Fischhoff,et al.  Evaluating the success of terror risk communications. , 2003, Biosecurity and bioterrorism : biodefense strategy, practice, and science.

[21]  Arkaitz Zubiaga,et al.  Detection and Resolution of Rumours in Social Media , 2017, ACM Comput. Surv..

[22]  J Brian Houston,et al.  Social media and disasters: a functional framework for social media use in disaster planning, response, and research. , 2015, Disasters.

[23]  Adam Acar,et al.  Twitter for crisis communication: lessons learned from Japan's tsunami disaster , 2011, Int. J. Web Based Communities.

[24]  Misako Takayasu,et al.  Rumor Diffusion and Convergence during the 3.11 Earthquake: A Twitter Case Study , 2015, PloS one.

[25]  Babak Abedin,et al.  Characterization of the Use of Social Media in Natural Disasters: A Systematic Review , 2014, 2014 IEEE Fourth International Conference on Big Data and Cloud Computing.

[26]  Jun Zhuang,et al.  Big Data and Predictive Analytics in Fire Risk Using Weather Data , 2020, Risk analysis : an official publication of the Society for Risk Analysis.

[27]  R. Procter,et al.  Reading the riots on Twitter: methodological innovation for the analysis of big data , 2013 .

[28]  Yoonsuh Jung Multiple predicting K-fold cross-validation for model selection , 2018 .

[29]  Ee-Peng Lim,et al.  The Retransmission of Rumor and Rumor Correction Messages on Twitter , 2017 .

[30]  Isabelle Augenstein,et al.  Turing at SemEval-2017 Task 8: Sequential Approach to Rumour Stance Classification with Branch-LSTM , 2017, *SEMEVAL.

[31]  Barbara Poblete,et al.  Twitter under crisis: can we trust what we RT? , 2010, SOMA '10.

[32]  Sinan Aral,et al.  The spread of true and false news online , 2018, Science.

[33]  H. Raghav Rao,et al.  Community Intelligence and Social Media Services: A Rumor Theoretic Analysis of Tweets During Social Crises , 2013, MIS Q..

[34]  Kate Starbird,et al.  How Information Snowballs: Exploring the Role of Exposure in Online Rumor Propagation , 2016, CSCW.

[35]  Zili Zhang,et al.  Predictors of the authenticity of Internet health rumours. , 2015, Health information and libraries journal.

[36]  Huaye Li,et al.  Computing the Veracity of Information through Crowds: A Method for Reducing the Spread of False Messages on Social Media , 2015, 2015 48th Hawaii International Conference on System Sciences.

[37]  Michelle M. Maresh-Fuehrer,et al.  Social media mapping innovations for crisis prevention, response, and evaluation , 2016, Comput. Hum. Behav..

[38]  Bairong Wang,et al.  Misinformation debunking and cross-platform information sharing through Twitter during Hurricanes Harvey and Irma: a case study on shelters and ID checks , 2020, Natural Hazards.

[39]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..