Wildland Fire Detection and Monitoring using a Drone-collected RGB/IR Image Dataset

Drone-based Unmanned Aerial Systems (UAS) provide an efficient means for early detection and monitoring of remote wildland fires due to their rapid deployment, low flight altitudes, high 3D maneuverability, and ever-expanding sensor capabilities. Recent sensor advancements have made side-by-side RGB/IR sensing feasible for UASs. The aggregation of optical and thermal images enables robust environmental observation, as the thermal feed provides information that would otherwise be obscured in a purely RGB setup, effectively "seeing through" thick smoke and tree occlusion. In this work, we present Fire detection and modeling: Aerial Multi-spectral image dataset (FLAME 2) [1], the first ever labeled collection of UAS-collected side-by-side RGB/IR aerial imagery of prescribed burns. Using FLAME 2, we then present two image-processing methodologies with Multi-modal Learning on our new dataset: (1) Deep Learning (DL)-based benchmarks for detecting fire and smoke frames with Transfer Learning and Feature Fusion. (2) an exemplary image-processing system cascaded in the DL-based classifier to perform fire localization. We show these two techniques achieve reasonable gains than either single-domain video inputs or training models from scratch in the fire detection task.

[1]  Alfian Abdul Halin,et al.  Fire-Net: A Deep Learning Framework for Active Forest Fire Detection , 2022, J. Sensors.

[2]  Ran Huang,et al.  Real-Time Flame Segmentation based on RGB-Thermal Fusion , 2021, 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[3]  Hafed Zarzour,et al.  A review on early wildfire detection from unmanned aerial vehicles using deep learning-based computer vision algorithms , 2021, Signal Process..

[4]  Abolfazl Razi,et al.  Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset , 2020, Comput. Networks.

[5]  Alexandra Moutinho,et al.  Wildfire detection using transfer learning on augmented datasets , 2020, Expert Syst. Appl..

[6]  Fatemeh Afghah,et al.  Transfer Learning for Wildfire Identification in UAV Imagery , 2020, 2020 54th Annual Conference on Information Sciences and Systems (CISS).

[7]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[8]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[9]  Andrei A. Rusu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[10]  Derek Hoiem,et al.  Learning without Forgetting , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.