A Multimodal Approach in Estimating Road Passability Through a Flooded Area Using Social Media and Satellite Images

This paper presents the algorithms that CERTH-ITI team deployed to tackle flood detection and road passability from social media and satellite data. Computer vision and deep learning techniques are combined in order to analyze social media and satellite images, while word2vec is used to analyze textual data. Multimodal fusion is also deployed in CERTH-ITI framework, both in early and late stage, by combining deep representation features in the former and semantic logic in the latter so as to provide a deeper and more meaningful understanding of the flood events.

[1]  Benjamin Bischke,et al.  The Multimedia Satellite Task at MediaEval 2018: Emergency Response for Flooding Events , 2018 .

[2]  Yiannis Kompatsiaris,et al.  Visual and Textual Analysis of Social Media and Satellite Images for Flood Detection @ Multimedia Satellite Task MediaEval 2017 , 2017, MediaEval.

[3]  Pengfei Zhang,et al.  Multimodal fusion for sensor data using stacked autoencoders , 2015, 2015 IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[6]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Ioannis Patras,et al.  Comparison of Fine-Tuning and Extension Strategies for Deep Convolutional Neural Networks , 2017, MMM.

[10]  Yiannis Kompatsiaris,et al.  People and Vehicles in Danger - A Fire and Flood Detection System in Social Media , 2018, 2018 IEEE 13th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP).

[11]  Yiannis Kompatsiaris,et al.  A Deep Neural Network for Oil Spill Semantic Segmentation in Sar Images , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).