Deep-Learning-Based Aerial Image Classification for Emergency Response Applications Using Unmanned Aerial Vehicles

Unmanned Aerial Vehicles (UAVs), equipped with camera sensors can facilitate enhanced situational awareness for many emergency response and disaster management applications since they are capable of operating in remote and difficult to access areas. In addition, by utilizing an embedded platform and deep learning UAVs can autonomously monitor a disaster stricken area, analyze the image in real-time and alert in the presence of various calamities such as collapsed buildings, flood, or fire in order to faster mitigate their effects on the environment and on human population. To this end, this paper focuses on the automated aerial scene classification of disaster events from on-board a UAV. Specifically, a dedicated Aerial Image Database for Emergency Response (AIDER) applications is introduced and a comparative analysis of existing approaches is performed. Through this analysis a lightweight convolutional neural network (CNN) architecture is developed, capable of running efficiently on an embedded platform achieving ~3x higher performance compared to existing models with minimal memory requirements with less than 2% accuracy drop compared to the state-of-the-art. These preliminary results provide a solid basis for further experimentation towards real-time aerial image classification for emergency response applications using UAVs.

[1]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Arturo de la Escalera,et al.  Survey of computer vision algorithms and applications for unmanned aerial vehicles , 2018, Expert Syst. Appl..

[3]  Ying Wang,et al.  A retrospective evaluation of energy-efficient object detection solutions on embedded devices , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[4]  Minsuk Kahng,et al.  Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers , 2018, IEEE Transactions on Visualization and Computer Graphics.

[5]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Anastasia Stratigea,et al.  Coping with Sustainability Objectives in Small and Medium-sized Cities and Island Communities , 2017 .

[7]  Yi Zhao,et al.  Saliency Detection and Deep Learning-Based Wildfire Identification in UAV Imagery , 2018, Sensors.

[8]  P. Petrides,et al.  Towards a holistic performance evaluation framework for drone-based object detection , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[9]  Ole-Christoffer Granmo,et al.  Deep Convolutional Neural Networks for Fire Detection in Images , 2017, EANN.

[10]  Shafiq R. Joty,et al.  Applications of Online Deep Learning for Crisis Response Using Social Media Information , 2016, ArXiv.

[11]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Farid Melgani,et al.  A Convolutional Neural Network Approach for Assisting Avalanche Search and Rescue Operations with UAV Imagery , 2017, Remote. Sens..

[13]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[14]  Theocharis Theocharides,et al.  Disaster Prevention and Emergency Response Using Unmanned Aerial Systems , 2017 .

[15]  Muhammad Shafique,et al.  An overview of next-generation architectures for machine learning: Roadmap, opportunities and challenges in the IoT era , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[16]  Feiping Nie,et al.  LRAGE: Learning Latent Relationships With Adaptive Graph Embedding for Aerial Scene Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[17]  M. N. Sulaiman,et al.  A Review On Evaluation Metrics For Data Classification Evaluations , 2015 .

[18]  Hyun-Woo Lee,et al.  Forest fire monitoring system based on aerial image , 2016, 2016 3rd International Conference on Information and Communication Technologies for Disaster Management (ICT-DM).

[19]  Vivienne Sze,et al.  Hardware for machine learning: Challenges and opportunities , 2017, 2017 IEEE Custom Integrated Circuits Conference (CICC).

[20]  Andreas Kamilaris,et al.  Disaster Monitoring using Unmanned Aerial Vehicles and Deep Learning , 2018, ArXiv.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Theocharis Theocharides,et al.  Optimized vision-directed deployment of UAVs for rapid traffic monitoring , 2018, 2018 IEEE International Conference on Consumer Electronics (ICCE).

[24]  E. Valuations.  A REVIEW ON EVALUATION METRICS FOR DATA CLASSIFICATION EVALUATIONS , 2015 .

[25]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[26]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[27]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yassine Hadjadj-Aoul.  2015 2nd International Conference on Information and Communication Technologies for Disaster Management (ICT-DM) , 2015 .

[29]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.