A Collaborative Region Detection and Grading Framework for Forest Fire Smoke Using Weakly Supervised Fine Segmentation and Lightweight Faster-RCNN

Forest fires are serious disasters that affect countries all over the world. With the progress of image processing, numerous image-based surveillance systems for fires have been installed in forests. The rapid and accurate detection and grading of fire smoke can provide useful information, which helps humans to quickly control and reduce forest losses. Currently, convolutional neural networks (CNN) have yielded excellent performance in image recognition. Previous studies mostly paid attention to CNN-based image classification for fire detection. However, the research of CNN-based region detection and grading of fire is extremely scarce due to a challenging task which locates and segments fire regions using image-level annotations instead of inaccessible pixel-level labels. This paper presents a novel collaborative region detection and grading framework for fire smoke using a weakly supervised fine segmentation and a lightweight Faster R-CNN. The multi-task framework can simultaneously implement the early-stage alarm, region detection, classification, and grading of fire smoke. To provide an accurate segmentation on image-level, we propose the weakly supervised fine segmentation method, which consists of a segmentation network and a decision network. We aggregate image-level information, instead of expensive pixel-level labels, from all training images into the segmentation network, which simultaneously locates and segments fire smoke regions. To train the segmentation network using only image-level annotations, we propose a two-stage weakly supervised learning strategy, in which a novel weakly supervised loss is proposed to roughly detect the region of fire smoke, and a new region-refining segmentation algorithm is further used to accurately identify this region. The decision network incorporating a residual spatial attention module is utilized to predict the category of forest fire smoke. To reduce the complexity of the Faster R-CNN, we first introduced a knowledge distillation technique to compress the structure of this model. To grade forest fire smoke, we used a 3-input/1-output fuzzy system to evaluate the severity level. We evaluated the proposed approach using a developed fire smoke dataset, which included five different scenes varying by the fire smoke level. The proposed method exhibited competitive performance compared to state-of-the-art methods.

[1]  Aymen Mouelhi,et al.  An Efficient Smoke Detection Algorithm Based on Deep Belief Network Classifier Using Energy and Intensity Features , 2020, Electronics.

[2]  Andrei B. Utkin,et al.  Development of neural network committee machines for automatic forest fire detection using lidar , 2004, Pattern Recognit..

[3]  Mohamed Elhoseny,et al.  Efficient Fire Detection for Uncertain Surveillance Environment , 2019, IEEE Transactions on Industrial Informatics.

[4]  Zhi-Hua Zhou,et al.  A brief introduction to weakly supervised learning , 2018 .

[5]  Aaron C. Courville,et al.  Generative adversarial networks , 2020 .

[6]  Nilton Cesar Fiedler,et al.  Applying GIS to develop a model for forest fire risk: A case study in Espírito Santo, Brazil. , 2016, Journal of environmental management.

[7]  Joseph Homer Saleh,et al.  Tackling Small Data Challenges in Visual Fire Detection: A Deep Convolutional Generative Adversarial Network Approach , 2021, IEEE Access.

[8]  Sung Wook Baik,et al.  Efficient Deep CNN-Based Fire Detection and Localization in Video Surveillance Applications , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[9]  Peng Lu,et al.  A Two-Stream CNN Model with Adaptive Adjustment of Receptive Field Dedicated to Flame Region Detection , 2021, Symmetry.

[10]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[11]  Y. I. Cho,et al.  An Efficient Deep Learning Algorithm for Fire and Smoke Detection with Limited Data , 2018 .

[12]  Gurjot Singh Gaba,et al.  Design of Multi-Information Fusion Based Intelligent Electrical Fire Detection System for Green Buildings , 2021, Sustainability.

[13]  F. Sunar,et al.  Evaluation of forest fire risk in the Mediterranean Turkish forests: A case study of Menderes region, Izmir , 2020 .

[14]  Liang Xu,et al.  A machine-vision inspection system for conveying attitudes of columnar objects in packing processes , 2016 .

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Shin-Juh Chen,et al.  Fire detection using smoke and gas sensors , 2007 .

[17]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[18]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[19]  Nojun Kwak,et al.  StackNet: Stacking feature maps for Continual learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[21]  Mark W. Schmidt,et al.  Where are the Blobs: Counting by Localization with Point Supervision , 2018, ECCV.

[22]  Ning Li,et al.  Development of an early warning fire detection system based on a laser spectroscopic carbon monoxide sensor using a 32-bit system-on-chip , 2019, Infrared Physics & Technology.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Sung Wook Baik,et al.  Convolutional Neural Networks Based Fire Detection in Surveillance Videos , 2018, IEEE Access.

[25]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[27]  L. Pessanha,et al.  Characterizing the paper industry sludge for environmentally-safe disposal. , 2019, Waste management.

[28]  Yi Zhao,et al.  Saliency Detection and Deep Learning-Based Wildfire Identification in UAV Imagery , 2018, Sensors.

[29]  Shifeng Zhang,et al.  RefineDet++: Single-Shot Refinement Neural Network for Object Detection , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Bernt Schiele,et al.  Simple Does It: Weakly Supervised Instance and Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Ahmad A. A. Alkhatib A Review on Forest Fire Detection Techniques , 2014, Int. J. Distributed Sens. Networks.

[33]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[34]  Feiniu Yuan,et al.  A Deep Normalization and Convolutional Neural Network for Image Smoke Detection , 2017, IEEE Access.

[35]  A. Enis Çetin,et al.  Covariance matrix-based fire and flame detection method in video , 2012, Machine Vision and Applications.

[36]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[37]  Xiaobo Lu,et al.  Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features , 2018, Multimedia Tools and Applications.

[38]  Sudhakar Sengan,et al.  Unmanned Aerial Vehicle (UAV) based Forest Fire Detection and monitoring for reducing false alarms in forest-fires , 2020, Comput. Commun..

[39]  Debarati Guha-Sapir,et al.  Estimating populations affected by disasters : A review of methodological issues and research gaps , 2015 .

[40]  Noel Varela,et al.  Wireless sensor network for forest fire detection , 2020, FNC/MobiSPC.

[41]  Jure Skvarč,et al.  Segmentation-based deep-learning approach for surface-defect detection , 2019, Journal of Intelligent Manufacturing.

[42]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[43]  Yung-Yu Chuang,et al.  Weakly Supervised Instance Segmentation using the Bounding Box Tightness Prior , 2019, NeurIPS.

[44]  Ronan Collobert,et al.  From image-level to pixel-level labeling with Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Lin Cao,et al.  A Forest Fire Detection System Based on Ensemble Learning , 2021, Forests.

[46]  Trevor Darrell,et al.  Fully Convolutional Multi-Class Multiple Instance Learning , 2014, ICLR.

[47]  Yang Liu,et al.  Forest Fire Smoke Recognition Based on Anchor Box Adaptive Generation Method , 2021 .

[48]  Jungpil Shin,et al.  Smoke Object Segmentation and the Dynamic Growth Feature Model for Video-Based Smoke Detection Systems , 2020, Symmetry.

[49]  Ole-Christoffer Granmo,et al.  Deep Convolutional Neural Networks for Fire Detection in Images , 2017, EANN.

[50]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[51]  Hasan Demirel,et al.  Fire detection in video sequences using a generic color model , 2009 .

[52]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[53]  Jiufeng Li,et al.  A novel optimized repeatedly random undersampling for selecting negative samples: A case study in an SVM-based forest fire susceptibility assessment. , 2020, Journal of environmental management.

[54]  Luís Torgo,et al.  A Survey of Predictive Modeling on Imbalanced Domains , 2016, ACM Comput. Surv..

[55]  Xiuxi Li,et al.  A Hybrid Character Recognition Approach Using Fuzzy Logic and Stroke Bayesian Program Learning With Naïve Bayes in Industrial Environments , 2020, IEEE Access.

[56]  Markus Loepfe,et al.  An image processing technique for fire detection in video images , 2006 .