Convolutional Neural Networks for Disaster Images Retrieval

This paper presents the method proposed by MRLDCSE team for the disaster image retrieval task in Mediaeval 2017 challenge on Multimedia and Satellite. In the proposed work, for visual information, we rely on Convolutional Neural Networks (CNN) features extracted with two different models pre-trained on ImageNet and places datasets. Moreover, a late fusion technique is employed to jointly utilize visual and the additional information available in the form of meta-data for the retrieval of disaster images from social media. The average precision for our three different runs with visual information only, meta-data and combination of meta-data and visual information are 95.73%, 18.23% and 92.55%, respectively.

[1]  Michael Riegler,et al.  JORD: A System for Collecting Information and Monitoring Natural Disasters by Linking Social Media with Satellite Imagery , 2017, CBMI.

[2]  Francesco G. B. De Natale,et al.  USED: a large-scale social event detection dataset , 2016, MMSys.

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Ashbindu Singh,et al.  Review Article Digital change detection techniques using remotely-sensed data , 1989 .

[5]  Francesco G. B. De Natale,et al.  A hierarchical approach to event discovery from single images using MIL framework , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[6]  Andreas Dengel,et al.  Contextual Enrichment of Remote-Sensed Events with Social Media Streams , 2016, ACM Multimedia.

[7]  Nicu Sebe,et al.  Event-based media processing and analysis: A survey of the literature , 2016, Image Vis. Comput..

[8]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[10]  Leysia Palen,et al.  Twitter adoption and use in mass convergence and emergency events , 2009 .

[11]  Chenliang Li,et al.  Twevent: segment-based event detection from tweets , 2012, CIKM.

[12]  Michael Riegler,et al.  The JORD System: Linking Sky and Social Multimedia Data to Natural Disasters , 2017, ICMR.

[13]  Hyeran Byun,et al.  Applications of Support Vector Machines for Pattern Recognition: A Survey , 2002, SVM.

[14]  Andreas Dengel,et al.  The Multimedia Satellite Task at MediaEval 2018 , 2017, MediaEval.

[15]  Sergey V. Samsonov,et al.  A review of the status of satellite remote sensing and image processing techniques for mapping natural hazards and disasters , 2009 .