A global database of historic and real-time flood events based on social media

Early event detection and response can significantly reduce the societal impact of floods. Currently, early warning systems rely on gauges, radar data, models and informal local sources. However, the scope and reliability of these systems are limited. Recently, the use of social media for detecting disasters has shown promising results, especially for earthquakes. Here, we present a new database for detecting floods in real-time on a global scale using Twitter. The method was developed using 88 million tweets, from which we derived over 10,000 flood events (i.e., flooding occurring in a country or first order administrative subdivision) across 176 countries in 11 languages in just over four years. Using strict parameters, validation shows that approximately 90% of the events were correctly detected. In countries where the first official language is included, our algorithm detected 63% of events in NatCatSERVICE disaster database at admin 1 level. Moreover, a large number of flood events not included in NatCatSERVICE were detected. All results are publicly available on www.globalfloodmonitor.org. Measurement(s) Internet Social Media • geographic location • flood Technology Type(s) digital curation • computational modeling technique Factor Type(s) text • timestamp Sample Characteristic - Environment flood Sample Characteristic - Location Earth (planet) Measurement(s) Internet Social Media • geographic location • flood Technology Type(s) digital curation • computational modeling technique Factor Type(s) text • timestamp Sample Characteristic - Environment flood Sample Characteristic - Location Earth (planet) Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.11294915

[1]  Qunying Huang,et al.  Using Twitter for tasking remote-sensing data collection and damage assessment: 2013 Boulder flood case study , 2016 .

[2]  Liyuan Liu,et al.  On the Variance of the Adaptive Learning Rate and Beyond , 2019, ICLR.

[3]  Harry H. Hendon,et al.  Indonesian Rainfall Variability: Impacts of ENSO and Local Air-Sea Interaction , 2003 .

[4]  Andrea Marchetti,et al.  A framework for detecting unfolding emergencies using humans as sensors , 2016, SpringerPlus.

[5]  Jazmine Maldonado,et al.  Robust Detection of Extreme Events Using Twitter: Worldwide Earthquake Monitoring , 2018, IEEE Transactions on Multimedia.

[6]  J. Aerts,et al.  Declining vulnerability to river floods and the global benefits of adaptation , 2015, Proceedings of the National Academy of Sciences.

[7]  W. Riley,et al.  Algorithms for frequency jump detection , 2008 .

[8]  Wolfgang Kron,et al.  How to deal properly with a natural catastrophe database – analysis of flood losses , 2012 .

[9]  Andrea Marchetti,et al.  EARS (earthquake alert and report system): a real time decision support system for earthquake crisis management , 2014, KDD.

[10]  Mehmet A. Orgun,et al.  A survey on real-time event detection from the Twitter data stream , 2018, J. Inf. Sci..

[11]  Alexander Zipf,et al.  Exploring the Geographical Relations Between Social Media and Flood Phenomena to Improve Situational Awareness - A Study About the River Elbe Flood in June 2013 , 2014, AGILE Conf..

[12]  B. Hurk,et al.  Forecast-based financing: an approach for catalyzing humanitarian action based on extreme weather and climate forecasts , 2014 .

[13]  Ed H. Chi,et al.  Language Matters In Twitter: A Large Scale Study , 2011, ICWSM.

[14]  Wael Khreich,et al.  A Survey of Techniques for Event Detection in Twitter , 2015, Comput. Intell..

[15]  Rudy Arthur,et al.  Social sensing of floods in the UK , 2017, PloS one.

[16]  Brenden Jongman,et al.  Early Flood Detection for Rapid Humanitarian Response: Harnessing Near Real-Time Satellite and Twitter Signals , 2015, ISPRS Int. J. Geo Inf..

[17]  Peter Salamon,et al.  Evaluation of the satellite-based Global Flood Detection System for measuring river discharge: influence of local factors , 2014 .

[18]  Yutaka Matsuo,et al.  Tweet Analysis for Real-Time Event Detection and Earthquake Reporting System Development , 2013, IEEE Transactions on Knowledge and Data Engineering.

[19]  F. Messner,et al.  FLOOD DAMAGE, VULNERABILITY AND RISK PERCEPTION – CHALLENGES FOR FLOOD DAMAGE RESEARCH , 2006 .

[20]  G. Moustakides,et al.  State-of-the-Art in Bayesian Changepoint Detection , 2010 .

[21]  Judith Gelernter,et al.  An algorithm for local geoparsing of microtext , 2013, GeoInformatica.

[22]  Joel C. Gill,et al.  Reviewing and visualizing the interactions of natural hazards , 2014 .

[23]  Mehmet A. Orgun,et al.  TwitterNews: Real time event detection from the Twitter data stream , 2016, PeerJ Prepr..

[24]  Jaime Campos,et al.  Domain-Independent Detection of Emergency Situations Based on Social Activity Related to Geolocations , 2018, WebSci.

[25]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[26]  Xun Wang,et al.  Real Time Event Detection in Twitter , 2013, WAIM.

[27]  Brenden Jongman,et al.  TAGGS: Grouping Tweets to Improve Global Geotagging for Disaster Response , 2017 .

[28]  Carlo Ratti,et al.  Geo-located Twitter as proxy for global mobility patterns , 2013, Cartography and geographic information science.

[29]  Jon M. Kleinberg,et al.  Bursty and Hierarchical Structure in Streams , 2002, Data Mining and Knowledge Discovery.

[30]  Stuart E. Middleton,et al.  Real-Time Crisis Mapping of Natural Disasters Using Social Media , 2014, IEEE Intelligent Systems.

[31]  L. Feyen,et al.  Development and evaluation of a framework for global flood hazard mapping , 2016 .

[32]  Jan Cools,et al.  Lessons from flood early warning systems , 2016 .

[33]  Yiannis Kompatsiaris,et al.  Location Extraction from Social Media , 2018, ACM Trans. Inf. Syst..

[34]  Florian Pappenberger,et al.  Technical review of large-scale hydrological models for implementation in operational flood forecasting schemes on continental level , 2016, Environ. Model. Softw..

[35]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[36]  Florian Pappenberger,et al.  On the Use of Global Flood Forecasts and Satellite-Derived Inundation Maps for Flood Monitoring in Data-Sparse Regions , 2015, Remote. Sens..

[37]  Geert-Jan Houben,et al.  Groundhog day: near-duplicate detection on Twitter , 2013, WWW.

[38]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[39]  Barbara Poblete,et al.  On-line relevant anomaly detection in the Twitter stream: an efficient bursty keyword detection model , 2013, ODD '13.

[40]  Sven Schade,et al.  Connecting a Digital Europe Through Location and Place , 2014, AGILE Conf..

[41]  Alessio Bosca,et al.  Early detection and information extraction for weather-induced floods using social media streams , 2018, International Journal of Disaster Risk Reduction.

[42]  J. S. Verkade,et al.  Estimation of Predictive Hydrological Uncertainty using Quantile Regression , 2010 .

[43]  Maurizio Tesconi,et al.  GSP (Geo-Semantic-Parsing): Geoparsing and Geotagging with Machine Learning on Top of Linked Data , 2018, ESWC.

[44]  Prakhar Gupta,et al.  Learning Word Vectors for 157 Languages , 2018, LREC.