Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset

Wildfires are one of the costliest and deadliest natural disasters in the US, causing damage to millions of hectares of forest resources and threatening the lives of people and animals. Of particular importance are risks to firefighters and operational forces, which highlights the need for leveraging technology to minimize danger to people and property. FLAME (Fire Luminosity Airbornebased Machine learning Evaluation) offers a dataset of aerial images of fires along with methods for fire detection and segmentation which can help firefighters and researchers to develop optimal fire management strategies. This paper provides a fire image dataset collected by drones during a prescribed burning piled detritus in an Arizona pine forest. The dataset includes video recordings and thermal heatmaps captured by infrared cameras. The captured videos and images are annotated, and labeled frame-wise to help researchers easily apply their fire detection and modeling algorithms. The paper also highlights solutions to two machine learning problems: (1) Binary classification of video frames based on the presence [and absence] of fire flames. An Artificial Neural Network (ANN) method is developed that achieved a 76% classification accuracy. (2) Fire detection using segmentation methods to precisely determine fire borders. A deep learning method is designed based on the U-Net ∗Corresponding author Email address: Alireza_Shamsoshoara@nau.edu (Alireza Shamsoshoara) Preprint submitted to Journal of Computer Networks December 29, 2020 ar X iv :2 01 2. 14 03 6v 1 [ cs .C V ] 2 8 D ec 2 02 0 up-sampling and down-sampling approach to extract a fire mask from the video frames. Our FLAME method approached a precision of 92%, and recall of 84%. Future research will expand the technique for free burning broadcast fire using thermal images.

[1]  Yuanzhi Li,et al.  Convergence Analysis of Two-layer Neural Networks with ReLU Activation , 2017, NIPS.

[2]  Janice L. Coen,et al.  Transforming Wildfire Detection and Prediction Using New and Underused Sensor and Data Sources Integrated with Modeling , 2018, Handbook of Dynamic Data Driven Applications Systems.

[3]  Fatemeh Afghah,et al.  An Autonomous Spectrum Management Scheme for Unmanned Aerial Vehicle Networks in Disaster Relief Operations , 2019, IEEE Access.

[4]  Youmin Zhang,et al.  Aerial Images-Based Forest Fire Detection for Firefighting Using Optical Remote Sensing Techniques and Unmanned Aerial Vehicles , 2017, Journal of Intelligent & Robotic Systems.

[5]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[6]  Mohd Anwar,et al.  The Automatic Detection of Sensitive Data in Smart Homes , 2019, HCI.

[7]  Elizabeth S. Bentley,et al.  A Coalition Formation Approach to Coordinated Task Allocation in Heterogeneous UAV Networks , 2017, 2018 Annual American Control Conference (ACC).

[8]  Rekha Gupta,et al.  A Highly Accurate Fire Detection Method Using Discriminate Method , 2018, 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[9]  Fatemeh Afghah,et al.  Wildfire Spread Modeling with Aerial Image Processing , 2020, 2020 IEEE 21st International Symposium on "A World of Wireless, Mobile and Multimedia Networks" (WoWMoM).

[10]  Laurent Njilla,et al.  Cooperative Spectrum Sharing and Trust Management in IoT Networks , 2020, Modeling and Design of Secure Internet of Things.

[11]  Fatemeh Afghah,et al.  A reputation-based stackelberg game model to enhance secrecy rate in spectrum leasing to selfish IoT devices , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[12]  Hasan Demirel,et al.  Fire detection in video sequences using a generic color model , 2009 .

[13]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[15]  A. Kott,et al.  Modeling and Design of Secure Internet of Things , 2020 .

[16]  Liyanage C. De Silva,et al.  State of the art of smoke and fire detection using image processing , 2017 .

[17]  David J. Grymin,et al.  Risk Aware SUAS Path Planning in an Unstructured Wildfire Environment , 2020, American Control Conference.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Fatemeh Afghah,et al.  Leader-follower based coalition formation in large-scale UAV networks, a quantum evolutionary approach , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[20]  Fatemeh Afghah,et al.  Transfer Learning for Wildfire Identification in UAV Imagery , 2020, 2020 54th Annual Conference on Information Sciences and Systems (CISS).

[21]  Abolfazl Razi,et al.  A Solution for Dynamic Spectrum Management in Mission-Critical UAV Networks , 2019, ArXiv.

[22]  Alberto Ferreira de Souza,et al.  Handling pedestrians in self-driving cars using image tracking and alternative path generation with Frenét frames , 2019, Comput. Graph..

[23]  Peter Corcoran,et al.  Traffic Light Control Using Deep Policy-Gradient and Value-Function Based Reinforcement Learning , 2017, ArXiv.

[24]  Turgay Çelik,et al.  Fire and smoke detection without sensors: Image processing based approach , 2007, 2007 15th European Signal Processing Conference.

[25]  Iván Santos-González,et al.  Forest Fire Prevention, Detection, and Fighting Based on Fuzzy Logic and Wireless Sensor Networks , 2018, Complex..

[26]  Xiaojun Qi,et al.  A computer vision-based method for fire detection in color videos , 2009 .

[27]  Iftikhar Ahmad,et al.  Fire Detection Using Multi Color Space and Background Modeling , 2020, Fire Technology.

[28]  Fatemeh Afghah,et al.  Use of a quantum genetic algorithm for coalition formation in large-scale UAV networks , 2019, Ad Hoc Networks.

[29]  Abolfazl Razi,et al.  Distributed Cooperative Spectrum Sharing in UAV Networks Using Multi-Agent Reinforcement Learning , 2018, 2019 16th IEEE Annual Consumer Communications & Networking Conference (CCNC).

[30]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Ian F. Akyildiz,et al.  Help from the Sky: Leveraging UAVs for Disaster Management , 2017, IEEE Pervasive Computing.

[32]  Atul K. Jain,et al.  Global Carbon Budget 2019 , 2019, Earth System Science Data.

[33]  Abolfazl Razi,et al.  Wildfire Monitoring in Remote Areas using Autonomous Unmanned Aerial Vehicles , 2019, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[34]  BOUZIANE BRIK,et al.  Federated Learning for UAVs-Enabled Wireless Networks: Use Cases, Challenges, and Open Problems , 2020, IEEE Access.

[35]  Fatemeh Afghah,et al.  A Real-time Framework for Trust Monitoring in a Network of Unmanned Aerial Vehicles , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[36]  Yuxi Li,et al.  Deep Reinforcement Learning: An Overview , 2017, ArXiv.

[37]  Lokman Nor Anis Aneza,et al.  FIRE RECOGNITION USING RGB AND YCBCR COLOR SPACE , 2015 .

[38]  Gu-Yeon Wei,et al.  Benchmarking TPU, GPU, and CPU Platforms for Deep Learning , 2019, ArXiv.

[39]  Youmin Zhang,et al.  UAV-based forest fire detection and tracking using image processing techniques , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[40]  Mohd Anwar,et al.  Towards Improving Privacy Control for Smart Homes: A Privacy Decision Framework , 2018, 2018 16th Annual Conference on Privacy, Security and Trust (PST).

[41]  Nasser Mozayani,et al.  Learning to predict where to look in interactive environments using deep recurrent q-learning , 2016, ArXiv.

[42]  Harkiran Kaur,et al.  Fog-assisted IoT-enabled scalable network infrastructure for wildfire surveillance , 2019, J. Netw. Comput. Appl..