Using a combination of human insights and ‘deep learning’ for real-time disaster communication

Abstract Using social media during natural disasters has become commonplace globally. In the U.S., public social media platforms are often a go-to because people believe: the 9-1-1 system becomes overloaded during emergencies and that first responders will see their posts. While social media requests may help save lives, these posts are difficult to find because there is more noise on public social media than clear signals of who needs help. This study compares human-coded images posted during 2017's Hurricane Harvey to machine-learned ‘deep learning’ classification methods. Our framework for feature extraction uses the VGG-16 convolutional neural network/multilayer perceptron classifiers for classifying the urgency and time period for a given image. We find that our qualitative results showcase that unique disaster experiences are not always captured through machine-learned methods. These methods work together to parse through the high levels of non-relevant content on social media to find relevant content and requests.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Donghua Zhou,et al.  Remaining useful life estimation - A review on the statistical data driven approaches , 2011, Eur. J. Oper. Res..

[3]  Jing Li,et al.  Citizens Communicating Health Information: Urging Others in their Community to Seek Help During a Flood , 2018, ISCRAM.

[4]  Radha Poovendran,et al.  On the Limitation of Convolutional Neural Networks in Recognizing Negative Images , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[5]  Hyekyung Woo,et al.  Public Trauma after the Sewol Ferry Disaster: The Role of Social Media in Understanding the Public Mood , 2015, International journal of environmental research and public health.

[6]  Liza Potts,et al.  Social Media in Disaster Response: How Experience Architects Can Build for Participation , 2013 .

[7]  Jing Li,et al.  Social Media in Citizen-Led Disaster Response: Rescuer Roles, Coordination Challenges, and Untapped Potential , 2018, ISCRAM.

[8]  Patric R. Spence,et al.  Variability in Twitter Content Across the Stages of a Natural Disaster: Implications for Crisis Communication , 2015 .

[9]  Anupam Joshi,et al.  Faking Sandy: characterizing and identifying fake images on Twitter during Hurricane Sandy , 2013, WWW.

[10]  Mohammad A. AlAhmad,et al.  Achieving Security for Images by LSB and MD5 , 2012 .

[11]  M. Villegas,et al.  Image-Text Dataset Generation for Image Annotation and Retrieval ⋆ , 2012 .

[12]  Mariette Awad,et al.  Damage Identification in Social Media Posts using Multimodal Deep Learning , 2018, ISCRAM.

[13]  Bing Wu,et al.  Impacts of information from various sources on the evacuation decision-making process during no-notice evacuations in campus environment , 2019, Journal of Transportation Safety & Security.

[14]  D. Murthy,et al.  Social media processes in disasters: Implications of emergent technology use. , 2017, Social science research.

[15]  Jong-Hoon Oh,et al.  Aid is Out There: Looking for Help from Tweets during a Large Scale Disaster , 2013, ACL.

[16]  Anne Oeldorf-Hirsch,et al.  Examining Twitter Content of State Emergency Management During Hurricane Joaquin , 2018, Communication Research Reports.

[17]  Patric R. Spence,et al.  Expressions of risk awareness and concern through Twitter: On the utility of using the medium as an indication of audience needs , 2014, Comput. Hum. Behav..

[18]  Sidharth Muralidharan,et al.  Hope for Haiti: An analysis of Facebook and Twitter usage during the earthquake relief efforts , 2011 .

[19]  Fernando Diaz,et al.  Extracting information nuggets from disaster- Related messages in social media , 2013, ISCRAM.

[20]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[21]  Temitayo Olofinlua.  Twitter: social communication in the twitter age , 2019, Information, Communication & Society.

[22]  Deborah Bunker,et al.  Emergency management in the changing world of social media: Framing the research agenda with the stakeholders through engaged scholarship , 2019, Int. J. Inf. Manag..

[23]  Avijit Paul.  Identifying relevant information for emergency services from twitter in response to natural disaster , 2015 .

[24]  Amanda Lee Hughes,et al.  Social Media in Disaster Communication , 2018 .

[25]  Bartosz Krawczyk,et al.  Learning from imbalanced data: open challenges and future directions , 2016, Progress in Artificial Intelligence.

[26]  Gregory D. Saxton,et al.  Tweeting Social Change , 2014 .

[27]  David Michael Todd,et al.  Natural disaster response : lessons from evaluations of the World Bank and others , 2011 .

[28]  Carlos Castillo,et al.  AIDR: artificial intelligence for disaster response , 2014, WWW.

[29]  Thomas Martinetz,et al.  Deep convolutional neural networks as generic feature extractors , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[30]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[31]  Leysia Palen,et al.  Microblogging during two natural hazards events: what twitter may contribute to situational awareness , 2010, CHI.

[32]  A. Bruns,et al.  #qldfloods and @QPSMedia: Crisis Communication on Twitter in the 2011 South East Queensland Floods , 2012 .

[33]  Scott A. Longwell,et al.  TWITTER AND DISASTERS , 2013 .

[34]  Shari R. Veil,et al.  A Work-in-Process Literature Review: Incorporating Social Media in Risk and Crisis Communication , 2011 .

[35]  Dhiraj Murthy,et al.  Visual Social Media and Big Data. Interpreting Instagram Images Posted on Twitter , 2016 .

[36]  Jonathan Corpus Ong,et al.  Tweeting Supertyphoon Haiyan: Evolving Functions of Twitter during and after a Disaster Event , 2016, PloS one.

[37]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[38]  Larry J. King.  Social Media Use During Natural Disasters: An Analysis of Social Media Usage During Hurricanes Harvey and Irma , 2018 .

[39]  Wei Zhao,et al.  Research on the deep learning of the small sample data based on transfer learning , 2017 .

[40]  Aron Culotta,et al.  Tweedr: Mining twitter to inform disaster response , 2014, ISCRAM.

[41]  Tilly Gurman,et al.  Reaching the Global Community During Disasters: Findings From a Content Analysis of the Organizational Use of Twitter After the 2010 Haiti Earthquake , 2015, Journal of health communication.

[42]  Carter T. Butts,et al.  Warning tweets: serial transmission of messages during the warning phase of a disaster event , 2014 .

[43]  Keri K. Stephens,et al.  New Media for Crisis Communication: Opportunities for Technical Translation, Dialogue, and Stakeholder Responses , 2010 .

[44]  Keri K. Stephens,et al.  Organizational Communication in Emergencies: Using Multiple Channels and Sources to Combat Noise and Capture Attention , 2013 .

[45]  Matthew Johnson,et al.  Training an Emergency-Response Image Classifier on Signal Data , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[46]  Christos Douligeris,et al.  An information management system for the emergency management of hurricane disasters , 2001 .

[47]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[48]  Brandon Van Der Heide,et al.  A social network as information: The effect of system generated reports of connectedness on credibility on Twitter , 2012, Comput. Hum. Behav..