Flood severity mapping from Volunteered Geographic Information by interpreting water level from images containing people: a case study of Hurricane Harvey

Abstract With increasing urbanization, in recent years there has been a growing interest and need in monitoring and analyzing urban flood events. Social media, as a new data source, can provide real-time information for flood monitoring. The social media posts with locations are often referred to as Volunteered Geographic Information (VGI), which can reveal the spatial pattern of such events. Since more images are shared on social media than ever before, recent research focused on the extraction of flood-related posts by analyzing images in addition to texts. Apart from merely classifying posts as flood relevant or not, more detailed information, e.g. the flood severity, can also be extracted based on image interpretation. However, it has been less tackled and has not yet been applied for flood severity mapping. In this paper, we propose a novel three-step process to extract and map flood severity information. First, flood relevant images are retrieved with the help of pre-trained convolutional neural networks as feature extractors. Second, the images containing people are further classified into four severity levels by observing the relationship between body parts and their partial inundation, i.e. images are classified according to the water level with respect to different body parts, namely ankle, knee, hip, and chest. Lastly, locations of the Tweets are used for generating a map of estimated flood extent and severity. This process was applied to an image dataset collected during Hurricane Harvey in 2017, as a proof of concept. The results show that VGI can be used as a supplement to remote sensing observations for flood extent mapping and is beneficial, especially for urban areas, where the infrastructure is often occluding water. Based on the extracted water level information, an integrated overview of flood severity can be provided for the early stages of emergency response.

[1]  D. Alderson,et al.  International Conference on Hydroinformatics 8-1-2014 Model Validation Using Crowd-Sourced Data From A Large Pluvial Flood , 2017 .

[2]  Minh-Son Dao,et al.  A Context-Aware Late-Fusion Approach for Disaster Image Retrieval from Social Media , 2018, ICMR.

[3]  Michael Riegler,et al.  CNN and GAN Based Satellite and Social Media Data Fusion for Disaster Detection , 2017, MediaEval.

[4]  Bolei Zhou,et al.  Scene Parsing through ADE20K Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Heidi Kreibich,et al.  Social media as an information source for rapid flood inundation mapping , 2015 .

[6]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[7]  Christian Heipke,et al.  Crowdsourcing geospatial data , 2010 .

[8]  Vinh-Tiep Nguyen,et al.  Flood Level Prediction via Human Pose Estimation from Social Media Images , 2020, ICMR.

[9]  K. Mcdougall,et al.  The use of LiDAR and volunteered geographic information to map flood extents and inundation , 2012 .

[10]  Muhammad Hanif,et al.  Flood detection using Social Media Data and Spectral Regression based Kernel Discriminant Analysis , 2017, MediaEval.

[11]  Jérôme Le Coz,et al.  Crowdsourced data for flood hydrology: Feedback from recent citizen science projects in Argentina, France and New Zealand , 2016 .

[12]  Gennady L. Andrienko,et al.  Tracing the German centennial flood in the stream of tweets: first lessons learned , 2013, GEOCROWD '13.

[13]  Monika Sester,et al.  Ensembled Convolutional Neural Network Models for Retrieving Flood Relevant Tweets , 2018, MediaEval.

[14]  Joost van de Weijer,et al.  Multi-modal Deep Learning Approach for Flood Detection , 2017, MediaEval.

[15]  Francesco G. B. De Natale,et al.  A Comparative Study of Global and Deep Features for the Analysis of User-Generated Natural Disaster Related Images , 2018, 2018 IEEE 13th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP).

[16]  Xiaobin Wang,et al.  DM_NLP at SemEval-2018 Task 12: A Pipeline System for Toponym Resolution , 2019, *SEMEVAL.

[17]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Nicola Conci,et al.  Convolutional Neural Networks for Disaster Images Retrieval , 2017, MediaEval.

[19]  Christopher S Lowry,et al.  CrowdHydrology: Crowdsourcing Hydrologic Data and Engaging Citizen Scientists , 2013, Ground water.

[20]  Linda See,et al.  A Review of Citizen Science and Crowdsourcing in Applications of Pluvial Flooding , 2019, Front. Earth Sci..

[21]  Frédéric Maire,et al.  Flood Mapping with Convolutional Neural Networks Using Spatio-Contextual Pixel Information , 2019, Remote. Sens..

[22]  Wolfram Burgard,et al.  CMRNet: Camera to LiDAR-Map Registration , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[23]  Michael Riegler,et al.  Automatic detection of passable roads after floods in remote sensed and social media data , 2019, Signal Process. Image Commun..

[24]  Cewu Lu,et al.  Two-Class Weather Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Yiannis Kompatsiaris,et al.  Visual and Textual Analysis of Social Media and Satellite Images for Flood Detection @ Multimedia Satellite Task MediaEval 2017 , 2017, MediaEval.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Jianhua Gong,et al.  Urban Flood Mapping Based on Unmanned Aerial Vehicle Remote Sensing and Random Forest Classifier—A Case of Yuyao, China , 2015 .

[28]  Arnejan van Loenen,et al.  Harvesting Social Media for Generation of Near Real-time Flood Maps☆ , 2016 .

[29]  M. Mukaka,et al.  Statistics corner: A guide to appropriate use of correlation coefficient in medical research. , 2012, Malawi medical journal : the journal of Medical Association of Malawi.

[30]  Hans-Peter Kriegel,et al.  Density-Based Clustering in Spatial Databases: The Algorithm GDBSCAN and Its Applications , 1998, Data Mining and Knowledge Discovery.

[31]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[32]  D. Leibovici,et al.  Rapid flood inundation mapping using social media, remote sensing and topographic data , 2017, Natural Hazards.

[33]  Cuizhen Wang,et al.  Linking picture with text: tagging flood relevant tweets for rapid flood inundation mapping , 2019 .

[34]  Haldun Akoglu,et al.  User's guide to correlation coefficients , 2018, Turkish journal of emergency medicine.

[35]  Robert Power,et al.  Sina Weibo Incident Monitor and Chinese Disaster Microblogging Classification , 2015 .

[36]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[37]  Chi-Farn Chen,et al.  Satellite-based investigation of flood-affected rice cultivation areas in Chao Phraya River Delta, Thailand , 2013 .

[38]  Sandro Martinis,et al.  A fully automated TerraSAR-X based flood service , 2015 .

[39]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Martha Larson,et al.  Retrieving Social Flooding Images Based on Multimodal Information , 2017, MediaEval.

[41]  Luke S. Smith,et al.  Assessing the utility of social media as a data source for flood risk management using a real‐time modelling framework , 2017 .

[42]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[44]  Hartwig H. Hochmair,et al.  Positional Accuracy of Twitter and Instagram Images in Urban Environments , 2016 .

[45]  Diansheng Guo,et al.  A novel approach to leveraging social media for rapid flood mapping: a case study of the 2015 South Carolina floods , 2018 .

[46]  Joachim Denzler,et al.  Enhancing Flood Impact Analysis using Interactive Retrieval of Social Media Images , 2019, ArXiv.

[47]  M. Goodchild.  Citizens as sensors: the world of volunteered geography , 2007 .

[48]  Ioana Popescu,et al.  Citizen observations contributing to flood modelling: opportunities and challenges , 2017 .

[49]  Arkaitz Zubiaga,et al.  WISC at MediaEval 2017: Multimedia Satellite Task , 2017, MediaEval.

[50]  Cuizhen Wang,et al.  Reconstructing Flood Inundation Probability by Enhancing Near Real-Time Imagery With Real-Time Gauges and Tweets , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Huan Ning,et al.  A visual–textual fused approach to automated tagging of flood-related tweets during a flood event , 2018, Int. J. Digit. Earth.

[52]  Li Linlin,et al.  DM_NLP at SemEval-2018 Task 12: A Pipeline System for Toponym Resolution. , 2019, NAACL 2019.

[53]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[54]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  M. Hodgson,et al.  Prototyping a Social Media Flooding Photo Screening System Based on Deep Learning , 2020, ISPRS Int. J. Geo Inf..

[56]  Gang Yu,et al.  BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation , 2018, ECCV.

[57]  João Porto de Albuquerque,et al.  Flood Citizen Observatory: a crowdsourcing-based approach for flood risk management in Brazil , 2014, SEKE.

[58]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Avinash Prasad,et al.  Evaluation of NDWI and MNDWI for assessment of waterlogging by integrating digital elevation model and groundwater level , 2015 .

[60]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[61]  Pascal Perez,et al.  Crowdsourced social media data for disaster management: Lessons from the PetaJakarta.org project , 2019, Comput. Environ. Urban Syst..

[62]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[63]  Yun Chen,et al.  Sub-pixel flood inundation mapping from multispectral remotely sensed images based on discrete particle swarm optimization , 2015 .

[64]  Yu Li,et al.  Urban flood mapping with an active self-learning convolutional neural network based on TerraSAR-X intensity and interferometric coherence , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[65]  P. Chaudhary,et al.  FLOOD-WATER LEVEL ESTIMATION FROM SOCIAL MEDIA IMAGES , 2019, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[66]  G. Atkinson,et al.  “Did You Feel It?” Intensity Data: A Surprisingly Good Measure of Earthquake Ground Motion , 2007 .

[67]  Monika Sester,et al.  Extraction of Pluvial Flood Relevant Volunteered Geographic Information (VGI) by Deep Learning from User Generated Texts and Photos , 2018, ISPRS Int. J. Geo Inf..

[68]  Benjamin Bischke,et al.  The Multimedia Satellite Task at MediaEval 2018: Emergency Response for Flooding Events , 2018 .

[69]  Jorge Pereira,et al.  Assessing flood severity from georeferenced photos , 2019 .

[70]  Christopher P. Konrad,et al.  Effects of Urban Development on Floods , 2003 .

[71]  Lin Li,et al.  Data-Driven Flood Detection using Neural Networks , 2017, MediaEval.

[72]  Andreas Dengel,et al.  Detection of Flooding Events in Social Multimedia and Satellite Imagery using Deep Neural Networks , 2017, MediaEval.