A review on early wildfire detection from unmanned aerial vehicles using deep learning-based computer vision algorithms

Abstract Wildfire is one of the most critical natural disasters that threaten wildlands and forest resources. Traditional firefighting systems, which are based on ground crew inspection, have several limits and can expose firefighters’ lives to danger. Thus, remote sensing technologies have become one of the most demanded strategies to fight against wildfires, especially UAV-based remote sensing technologies. They have been adopted to detect forest fires at their early stages, before becoming uncontrollable. Autonomous wildfire early detection from UAV-based visual data using different deep learning algorithms has attracted significant interest in the last few years. To this end, in this paper, we focused on wildfires detection at their early stages in forest and wildland areas, using deep learning-based computer vision algorithms to prevent and then reduce disastrous losses in terms of human lives and forest resources.

[1]  M. Rodrigues,et al.  Has COVID-19 halted winter-spring wildfires in the Mediterranean? Insights for wildfire science under a pandemic context , 2020, Science of The Total Environment.

[2]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[4]  Susan I. Stewart,et al.  After the fire: Perceptions of land use planning to reduce wildfire risk in eight communities across the United States , 2020, International Journal of Disaster Risk Reduction.

[5]  Varun Totakura,et al.  Improved Safety of Self-Driving Car using Voice Recognition through CNN , 2021 .

[6]  Roman Larionov,et al.  Wildfire Segmentation on Satellite Images using Deep Learning , 2020, 2020 Moscow Workshop on Electronic and Networking Technologies (MWENT).

[7]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[8]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[9]  Masuda A. Tonima,et al.  Forest fire flame and smoke detection from UAV-captured images using fire-specific color features and multi-color space local binary pattern , 2020, Journal of Unmanned Vehicle Systems.

[10]  Matteo Bo,et al.  Urban air pollution, climate change and wildfires: The case study of an extended forest fire episode in northern Italy favoured by drought and warm weather conditions , 2020, Energy Reports.

[11]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Y. Kountouris.  Human activity, daylight saving time and wildfire occurrence. , 2020, The Science of the total environment.

[13]  Emre Selvi,et al.  Use of Fire-Extinguishing Balls for a Conceptual System of Drone-Assisted Wildfire Fighting , 2019, Drones.

[14]  Dimitrios Chrysostomou,et al.  A Framework for Wildfire Inspection Using Deep Convolutional Neural Networks , 2020, 2020 IEEE/SICE International Symposium on System Integration (SII).

[15]  C. Lawrence,et al.  The development and validation of the bushfire psychological preparedness scale , 2020 .

[16]  Sunghwan Kim,et al.  Fire Detection Using Video Images and Temporal Variations , 2019, 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[17]  Tania Stathaki,et al.  A Novel Framework for Early Fire Detection Using Terrestrial and Aerial 360-Degree Images , 2020, ACIVS.

[18]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Plamen Zahariev,et al.  Early Forest Fire Detection Using Drones and Artificial Intelligence , 2019, 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[20]  Steven Verstockt,et al.  Video fire detection - Review , 2013, Digit. Signal Process..

[21]  Mounir Sayadi,et al.  Early smoke detection of forest wildfire video using deep belief network , 2018, 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP).

[22]  Youmin Zhang,et al.  A UAV-based Forest Fire Detection Algorithm Using Convolutional Neural Network , 2018, 2018 37th Chinese Control Conference (CCC).

[23]  Alberto Viseras,et al.  Wildfire Monitoring and Hotspots Detection with Aerial Robots: Measurement Campaign and First Results , 2019, 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[24]  Zechao Li,et al.  Online Video Object Detection via Local and Mid-Range Feature Propagation , 2020, HUMA @ ACM Multimedia.

[25]  Rishabh Sharma,et al.  FireNet: A Specialized Lightweight Fire & Smoke Detection Model for Real-Time IoT Applications , 2019, ArXiv.

[26]  T. Ngo,et al.  Impact of Australia's catastrophic 2019/20 bushfire season on communities and environment. Retrospective analysis and current trends , 2020 .

[27]  Roman Solovyev,et al.  Weighted boxes fusion: Ensembling boxes from different object detection models , 2021, Image Vis. Comput..

[28]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  G. Kemper,et al.  WILDFIRE DETECTION AND DISASTER MONITORING SYSTEM USING UAS AND SENSOR FUSION TECHNOLOGIES , 2020 .

[30]  Johan Potgieter,et al.  Plant Disease Detection and Classification by Deep Learning , 2019, Plants.

[31]  Armin Gruen,et al.  Pre- and Post-Fire Comparison of Forest Areas in 3D , 2018, Intelligent Systems for Crisis Management.

[32]  Abolfazl Razi,et al.  Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset , 2020, Comput. Networks.

[33]  Byoung Chul Ko,et al.  Light-Weight Student LSTM for Real-Time Wildfire Smoke Detection , 2020, Sensors.

[34]  Mrs Vidya Zope,et al.  IoT Sensor and Deep Neural Network based Wildfire Prediction System , 2020, 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS).

[35]  Ming Zong,et al.  Motion saliency based multi-stream multiplier ResNets for action recognition , 2021, Image Vis. Comput..

[36]  Jinhui Tang,et al.  CTNet: Context-Based Tandem Network for Semantic Segmentation , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Qixing Zhang,et al.  Wildland Forest Fire Smoke Detection Based on Faster R-CNN using Synthetic Smoke Images , 2018 .

[38]  Nikolaos Grammalidis,et al.  Higher Order Linear Dynamical Systems for Smoke Detection in Video Surveillance Applications , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[40]  Ali El Moussati,et al.  Early Forest Fire Detection System using Wireless Sensor Network and Deep Learning , 2020 .

[41]  Mohit Dua,et al.  Fog Computing and Deep CNN Based Efficient Approach to Early Forest Fire Detection with Unmanned Aerial Vehicles , 2019, Inventive Computation Technologies.

[42]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[43]  Yichao Cao,et al.  An Attention Enhanced Bidirectional LSTM for Early Forest Fire Smoke Recognition , 2019, IEEE Access.

[44]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[45]  Yi Zhao,et al.  Saliency Detection and Deep Learning-Based Wildfire Identification in UAV Imagery , 2018, Sensors.

[46]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[47]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[48]  Xiaoqin Zhang,et al.  A Slight Smoke Perceptual Network , 2019, IEEE Access.

[49]  Y. Yamazaki,et al.  Forest Fires Mapping and Monitoring of current and past forest fire activity from Meteosat Second Generation Data , 2010, Environ. Model. Softw..

[50]  Plamen Zahariev,et al.  Application of Artificial Intelligence in UAV platforms for Early Forest Fire Detection , 2019, 2019 27th National Conference with International Participation (TELECOM).

[51]  Jiaolong Xu,et al.  Deep Convolutional Neural Networks for Forest Fire Detection , 2016 .

[52]  Hyun-Woo Lee,et al.  Deep neural networks for wild fire detection with unmanned aerial vehicle , 2017, 2017 IEEE International Conference on Consumer Electronics (ICCE).

[53]  S. Vardoulakis,et al.  Lessons Learned from the Australian Bushfires: Climate Change, Air Pollution, and Public Health. , 2020, JAMA internal medicine.

[54]  Lingxia Mu,et al.  A Deep Learning Based Forest Fire Detection Approach Using UAV and YOLOv3 , 2019, 2019 1st International Conference on Industrial Artificial Intelligence (IAI).

[55]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[56]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Nikos Grammalidis,et al.  A Review on Early Forest Fire Detection Systems Using Optical Remote Sensing , 2020, Sensors.

[58]  Ahmad A. A. Alkhatib.  A Review on Forest Fire Detection Techniques , 2014, Int. J. Distributed Sens. Networks.

[59]  Lingxia Mu,et al.  UAV Image-based Forest Fire Detection Approach Using Convolutional Neural Network , 2019, 2019 14th IEEE Conference on Industrial Electronics and Applications (ICIEA).

[60]  Lin Yu,et al.  Simulation of water and chemical transport of chloride from the forest ecosystem to the stream , 2021, Environ. Model. Softw..

[61]  Begoña C. Arrue,et al.  Computer vision techniques for forest fire perception , 2008, Image Vis. Comput..

[62]  Jinhui Tang,et al.  CAD: Scale Invariant Framework for Real-Time Object Detection , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[63]  Jürgen Beyerer,et al.  Low-resolution Convolutional Neural Networks for video face recognition , 2016, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[64]  C. Justice,et al.  The collection 6 MODIS active fire detection algorithm and fire products , 2016, Remote sensing of environment.

[65]  G. Sai Kiran,et al.  Development of a Surveillance System for Forest Fire Detection and Monitoring using Drones , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[66]  Anis Koubaa,et al.  Car Detection using Unmanned Aerial Vehicles: Comparison between Faster R-CNN and YOLOv3 , 2018, 2019 1st International Conference on Unmanned Vehicle Systems-Oman (UVS).

[67]  Carlton R. Pennypacker,et al.  Preliminary Results from a Wildfire Detection System Using Deep Learning on Remote Camera Images , 2020, Remote. Sens..

[68]  Nicolas Usunier,et al.  End-to-End Object Detection with Transformers , 2020, ECCV.

[69]  A. Enis Çetin,et al.  Early Wildfire Smoke Detection Based on Motion-based Geometric Image Transformation and Deep Convolutional Generative Adversarial Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[70]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[71]  Qiang Zhao,et al.  A New Model of RGB-D Camera Calibration Based on 3D Control Field , 2019, Sensors.

[72]  Nikolaos Grammalidis,et al.  Early Fire Detection Based on Aerial 360-Degree Sensors, Deep Convolution Neural Networks and Exploitation of Fire Dynamic Textures , 2020, Remote. Sens..

[73]  Miguel Almeida,et al.  Classification of potential fire outbreaks: A fuzzy modeling approach based on thermal images , 2019, Expert Syst. Appl..

[74]  Igor Pantiukhin,et al.  Analysis of Machine Learning Methods for Wildfire Security Monitoring with an Unmanned Aerial Vehicles , 2019, 2019 24th Conference of Open Innovations Association (FRUCT).

[75]  M. Emelko,et al.  Severe western Canadian wildfire affects water quality even at large basin scales. , 2020, Water research.

[76]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[77]  Plamen Zahariev,et al.  Emerging Methods for Early Detection of Forest Fires Using Unmanned Aerial Vehicles and Lorawan Sensor Networks , 2018, 2018 28th EAEEIE Annual Conference (EAEEIE).

[78]  Ahmed Kechida,et al.  A Survey on Lightweight CNN-Based Object Detection Algorithms for Platforms with Limited Computational Resources , 2019 .

[79]  Yong Xiao,et al.  Human tracking from single RGB-D camera using online learning , 2019, Image Vis. Comput..

[81]  Panagiotis G. Sarigiannidis,et al.  A Review on UAV-Based Applications for Precision Agriculture , 2019, Inf..

[82]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[83]  A. Enis Çetin,et al.  Deep Convolutional Generative Adversarial Networks for Flame Detection in Video , 2020, ICCCI.

[84]  Lingxia Mu,et al.  A YOLOv3-based Learning Strategy for Real-time UAV-based Forest Fire Detection , 2020, 2020 Chinese Control And Decision Conference (CCDC).

[85]  Fengju Bu,et al.  Intelligent and vision-based fire detection systems: A survey , 2019, Image Vis. Comput..

[86]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[87]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[88]  Impact of human factors on wildfire occurrence in Mississippi, United States , 2017 .