Weakly Supervised Forest Fire Segmentation in UAV Imagery Based on Foreground-Aware Pooling and Context-Aware Loss

In recent years, tragedies caused by forest fires have been frequently reported. Forest fires not only result in significant economic losses but also cause environmental damage. The utilization of computer vision techniques and unmanned aerial vehicles (UAVs) for forest fire monitoring has become a primary approach to accurately locate and extinguish fires during their early stages. However, traditional computer-based methods for UAV forest fire image segmentation require a large amount of pixel-level labeled data to train the networks, which can be time-consuming and costly to acquire. To address this challenge, we propose a novel weakly supervised approach for semantic segmentation of fire images in this study. Our method utilizes self-supervised attention foreground-aware pooling (SAP) and context-aware loss (CAL) to generate high-quality pseudo-labels, serving as substitutes for manual annotation. SAP collaborates with bounding box and class activation mapping (CAM) to generate a background attention map, which aids in the generation of accurate pseudo-labels. CAL further improves the quality of the pseudo-labels by incorporating contextual information related to the target objects, effectively reducing environmental noise. We conducted experiments on two publicly available UAV forest fire datasets: the Corsican dataset and the Flame dataset. Our proposed method achieved impressive results, with IoU values of 81.23% and 76.43% for the Corsican dataset and the Flame dataset, respectively. These results significantly outperform the latest weakly supervised semantic segmentation (WSSS) networks on forest fire datasets.

[1]  Lili Feng,et al.  The Forest Fire Dynamic Change Influencing Factors and the Impacts on Gross Primary Productivity in China , 2023, Remote. Sens..

[2]  X. Chen,et al.  Economic Loss Assessment and Spatial–Temporal Distribution Characteristics of Forest Fires: Empirical Evidence from China , 2022, Forests.

[3]  Dai Quoc Tran,et al.  Advanced wildfire detection using generative adversarial network-based augmented datasets and weakly supervised object localization , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[4]  Catarina Barata,et al.  Weakly Supervised Fire and Smoke Segmentation in Forest Images with CAM and CRF , 2022, 2022 26th International Conference on Pattern Recognition (ICPR).

[5]  Xing Wang,et al.  Semantic Segmentation of Fruits on Multi-sensor Fused Data in Natural Orchards , 2022, Comput. Electron. Agric..

[6]  W. Gonçalves,et al.  Road extraction in remote sensing data: A survey , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[7]  Zhaoyou Lu,et al.  Comparative Research on Forest Fire Image Segmentation Algorithms Based on Fully Convolutional Neural Networks , 2022, Forests.

[8]  Weiwei Cai,et al.  A high-precision forest fire smoke detection approach based on ARGNet , 2022, Comput. Electron. Agric..

[9]  W. Cui,et al.  Semantic Segmentation and Analysis on Sensitive Parameters of Forest Fire Smoke Using Smoke-Unet and Landsat-8 Imagery , 2021, Remote. Sens..

[10]  Jiali Shang,et al.  Automated delineation of agricultural field boundaries from Sentinel-2 images using recurrent residual U-Net , 2021, Int. J. Appl. Earth Obs. Geoinformation.

[11]  Yanfeng Wang,et al.  A High-precision Forest Fire Smoke Detection Approach Based on DRGNet to Remote Sensing Through Uavs , 2021 .

[12]  Ming-Hsuan Yang,et al.  Weakly Supervised Object Localization and Detection: A Survey , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Bumsub Ham,et al.  Background-Aware Pooling and Noise-Aware Loss for Weakly-Supervised Semantic Segmentation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Guosheng Lin,et al.  Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  In-Jae Yu,et al.  Puzzle-CAM: Improved Localization Via Matching Partial And Full Features , 2021, 2021 IEEE International Conference on Image Processing (ICIP).

[16]  L. Jorge,et al.  A Review on Deep Learning in UAV Remote Sensing , 2021, Int. J. Appl. Earth Obs. Geoinformation.

[17]  Myungjoo Kang,et al.  Semantic Fire Segmentation Model Based on Convolutional Neural Network for Outdoor Image , 2021, Fire Technology.

[18]  Abolfazl Razi,et al.  Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset , 2020, Comput. Networks.

[19]  Xilin Chen,et al.  Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Roman Larionov,et al.  Wildfire Segmentation on Satellite Images using Deep Learning , 2020, 2020 Moscow Workshop on Electronic and Networking Technologies (MWENT).

[21]  Dimitrios Chrysostomou,et al.  A Framework for Wildfire Inspection Using Deep Convolutional Neural Networks , 2020, 2020 IEEE/SICE International Symposium on System Integration (SII).

[22]  Yi Wang,et al.  Real-time forest smoke detection using hand-designed features and deep learning , 2019, Comput. Electron. Agric..

[23]  S. Dimitropoulos Fighting fire with science , 2019, Nature.

[24]  Jesper E. van Engelen,et al.  A survey on semi-supervised learning , 2019, Machine Learning.

[25]  Neil Flood,et al.  Using a U-net convolutional neural network to map woody vegetation extent from high resolution satellite imagery across Queensland, Australia , 2019, Int. J. Appl. Earth Obs. Geoinformation.

[26]  Quoc V. Le,et al.  Searching for MobileNetV3 , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Suha Kwak,et al.  Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Mert R. Sabuncu,et al.  Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels , 2018, NeurIPS.

[29]  Moulay A. Akhloufi,et al.  Computer vision for wildfire research: An evolving image dataset for processing and analysis , 2017 .

[30]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.