A survey on technologies for automatic forest fire monitoring, detection, and fighting using unmanned aerial vehicles and remote sensing techniques

Because of their rapid maneuverability, extended operational range, and improved personnel safety, unmanned aerial vehicles (UAVs) with vision-based systems have great potential for monitoring, detecting, and fighting forest fires. Over the last decade, UAV-based forest fire fighting technology has shown increasing promise. This paper presents a systematic overview of current progress in this field. First, a brief review of the development and system architecture of UAV systems for forest fire monitoring, detection, and fighting is provided. Next, technologies related to UAV forest fire monitoring, detection, and fighting are briefly reviewed, including those associated with fire detection, diagnosis, and prognosis, image vibration elimination, and cooperative control of UAVs. The final section outlines existing challenges and potential solutions in the application of UAVs to forest firefighting.

[1]  Dengyi Zhang,et al.  Forest fire and smoke detection based on video image segmentation , 2007, International Symposium on Multispectral Image Processing and Pattern Recognition.

[2]  George Pierce Jones,et al.  THE FEASIBILITY OF USING SMALL UNMANNED AERIAL VEHICLES FOR WILDLIFE RESEARCH , 2003 .

[3]  K. P. Schwarz,et al.  Digital image georeferencing from a multiple camera system by GPS/INS , 2001 .

[4]  J. Théau,et al.  Recent applications of unmanned aerial imagery in natural resource management , 2014 .

[5]  Chang-chun Li,et al.  Quick image-processing method of UAV without control points data in earthquake disaster area , 2011 .

[6]  GEORGE PIERCE JONES,et al.  An Assessment of Small Unmanned Aerial Vehicles for Wildlife Research , 2006 .

[7]  Penggen Cheng,et al.  Unmanned aerial vehicle (UAV) real-time video registration for forest fire monitoring , 2005, Proceedings. 2005 IEEE International Geoscience and Remote Sensing Symposium, 2005. IGARSS '05..

[8]  A. Enis Çetin,et al.  Contour based smoke detection in video using wavelets , 2006, 2006 14th European Signal Processing Conference.

[9]  Aníbal Ollero,et al.  A distributed architecture for a robotic platform with aerial sensor transportation and self‐deployment capabilities , 2011, J. Field Robotics.

[10]  Dengyi Zhang,et al.  SVM based forest fire detection using static and dynamic features , 2011, Comput. Sci. Inf. Syst..

[11]  A. Enis Çetin,et al.  Entropy-Functional-Based Online Adaptive Decision Fusion Framework With Application to Wildfire Detection in Video , 2012, IEEE Transactions on Image Processing.

[12]  Franck Guarnieri,et al.  Intelligent software agents for forest fire prevention and fighting , 2001 .

[13]  Luis Merino,et al.  Unmanned aerial vehicles as tools for forest-fire fighting , 2006 .

[14]  Watchara Chatwiriya,et al.  Forest Fire Smoke Detection in Video Based on Digital Image Processing Approach with Static and Dynamic Characteristic Analysis , 2011, 2011 First ACIS/JNU International Conference on Computers, Networks, Systems and Industrial Engineering.

[15]  Baisravan Homchaudhuri,et al.  Cooperative Control of Multiple Uninhabited Aerial Vehicles for Monitoring and Fighting Wildfires , 2011, J. Aerosp. Comput. Inf. Commun..

[16]  Chitra Dadkhah,et al.  Automatic fire detection based on soft computing techniques: review from 2000 to 2010 , 2012, Artificial Intelligence Review.

[17]  Youmin Zhang,et al.  UAV-based forest fire detection and tracking using image processing techniques , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[18]  Haiyang Chao,et al.  Cooperative Remote Sensing Using Multiple Unmanned Vehicles , 2012 .

[19]  Timothy W. McLain,et al.  Decentralized Cooperative Aerial Surveillance Using Fixed-Wing Miniature UAVs , 2006, Proceedings of the IEEE.

[20]  Stephen E. Dunagan,et al.  Demonstrating UAV-acquired real-time thermal data over fires , 2003 .

[21]  Hasan Demirel,et al.  Fire detection in video sequences using a generic color model , 2009 .

[22]  Zhao Liang,et al.  Application of low-altitude remote sensing image by unmanned airship in geological hazards investigation , 2012, 2012 5th International Congress on Image and Signal Processing.

[23]  Ke Xu,et al.  Review of fire detection technologies based on video image , 2013 .

[24]  Randal W. Beard,et al.  Cooperative forest fire monitoring using multiple UAVs , 2007, 2007 46th IEEE Conference on Decision and Control.

[25]  M Kumar,et al.  Optimal fireline generation for wildfire fighting in uncertain and heterogeneous environment , 2010, Proceedings of the 2010 American Control Conference.

[26]  Johan E. S. Fransson,et al.  Current and emerging operational uses of remote sensing in Swedish forestry , 2007 .

[27]  Jun Wu,et al.  Real-Time UAV Video Processing for Quick- Response to Natural Disaster , 2006, 2006 IEEE International Symposium on Geoscience and Remote Sensing.

[28]  Youmin Zhang,et al.  Bibliographical review on reconfigurable fault-tolerant control systems , 2003, Annu. Rev. Control..

[29]  A. Moccia,et al.  An Integrated Electro-Optical Payload System for Forest Fires Monitoring from Airborne Platform , 2007, 2007 IEEE Aerospace Conference.

[30]  Timo Pukkala,et al.  A fire probability model for forest stands in Catalonia (north-east Spain) , 2006 .

[31]  Aníbal Ollero,et al.  Motion compensation and object detection for autonomous helicopter visual navigation in the COMETS system , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[32]  Clark N. Taylor,et al.  Georeferenced Mosaics for Tracking Fires Using Unmanned Miniature Air Vehicles , 2011, J. Aerosp. Comput. Inf. Commun..

[33]  L. Vergara,et al.  Multisensor Network System for Wildfire Detection Using Infrared Image Processing , 2013, TheScientificWorldJournal.

[34]  C. Daughtry,et al.  REMOTE SENSING OF CROP LEAF AREA INDEX USING UNMANNED AIRBORNE VEHICLES , 2008 .

[35]  Youmin Zhang,et al.  A Distributed Deployment Strategy for a Network of Cooperative Autonomous Vehicles , 2015, IEEE Transactions on Control Systems Technology.

[36]  Francis Y. Enomoto,et al.  The Ikhana unmanned airborne system (UAS) western states fire imaging missions: from concept to reality (2006–2010) , 2011 .

[37]  Moulay A. Akhloufi,et al.  A 3D vision system for the measurement of the rate of spread and the height of fire fronts , 2010 .

[38]  Anoop M. Pullivelli,et al.  Stability analysis of low‐cost digital cameras for aerial mapping using different georeferencing techniques , 2006 .

[39]  Begoña C. Arrue,et al.  Computer vision techniques for forest fire perception , 2008, Image Vis. Comput..

[40]  Aníbal Ollero,et al.  Real-Time Feature Matching in Image Sequences for Non-Structured Environments. Applications to Vehicle Guidance , 2000, J. Intell. Robotic Syst..

[41]  V. Vipin.  Image Processing Based Forest Fire Detection , 2012 .

[42]  Jie Hou,et al.  Fire Detection Algorithms in Video Images for High and Large-Span Space Structures , 2009, 2009 2nd International Congress on Image and Signal Processing.

[43]  Sinan Kalkan,et al.  Vision-Based Detection and Distance Estimation of Micro Unmanned Aerial Vehicles , 2015, Sensors.

[44]  Klamer Schutte,et al.  Autonomous Forest Fire Detection , 1998 .

[45]  J. A. Olivas.  Forest fire prediction and management using soft computing , 2003, IEEE International Conference on Industrial Informatics, 2003. INDIN 2003. Proceedings..

[46]  Begoña C. Arrue,et al.  Techniques for reducing false alarms in infrared forest-fire automatic detection systems , 1999 .

[47]  Jian Wang,et al.  Multi-feature fusion based fast video flame detection , 2010 .

[48]  Martine De Cock,et al.  An evolved fuzzy logic system for fire size prediction , 2009, NAFIPS 2009 - 2009 Annual Meeting of the North American Fuzzy Information Processing Society.

[49]  Jie Hou,et al.  Fire detection algorithms for video images of large space structures , 2009, Multimedia Tools and Applications.

[50]  Tomàs Margalef,et al.  Applying a Dynamic Data Driven Genetic Algorithm to Improve Forest Fire Spread Prediction , 2008, ICCS.

[51]  Steven Verstockt,et al.  Video fire detection - Review , 2013, Digit. Signal Process..

[52]  Xiaojun Qi,et al.  A computer vision-based method for fire detection in color videos , 2009 .

[53]  A. Enis Çetin,et al.  Flame detection in video using hidden Markov models , 2005, IEEE International Conference on Image Processing 2005.

[54]  Sai-Ming Li,et al.  Forest fire monitoring with multiple small UAVs , 2005, Proceedings of the 2005, American Control Conference, 2005..

[55]  Zvi Boger,et al.  Infrared Flame Detection System Using Multiple Neural Networks , 2007, 2007 International Joint Conference on Neural Networks.

[56]  Michael Tranchitella,et al.  Using Tactical Unmanned Aerial Systems to Monitor and Map Wildfires , 2007 .

[57]  Juan Andrade-Cetto,et al.  Computing the rate of spread of linear flame fronts by thermal image processing , 2006 .

[58]  A. Enis Çetin,et al.  Computer vision based method for real-time fire and flame detection , 2006, Pattern Recognit. Lett..

[59]  Luis Merino,et al.  Cooperative Unmanned Aerial Systems for Fire Detection, Monitoring, and Extinguishing , 2015 .

[60]  Aníbal Ollero,et al.  A cooperative perception system for multiple UAVs: Application to automatic detection of forest fires , 2006, J. Field Robotics.

[61]  Juan López,et al.  Architecture for a helicopter-based unmanned aerial systems wildfire surveillance system , 2011 .

[62]  Kimon P. Valavanis,et al.  A UAV vision system for airborne surveillance , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[63]  Donald G. Leckie,et al.  Advances in remote sensing technologies for forest surveys and management , 1990 .

[64]  Pablo J. Zarco-Tejada,et al.  Thermal and Narrowband Multispectral Remote Sensing for Vegetation Monitoring From an Unmanned Aerial Vehicle , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[65]  Aníbal Ollero,et al.  Multiple eyes in the skies: architecture and perception issues in the COMETS unmanned air vehicles project , 2005, IEEE Robotics & Automation Magazine.

[66]  Aníbal Ollero,et al.  Journal of Intelligent & Robotic Systems manuscript No. (will be inserted by the editor) Experimental Results in Multi-UAV Coordination for Disaster Management and Civil Security Applications , 2022 .

[67]  Ioannis Karafyllidis,et al.  Design of a dedicated parallel processor for the prediction of forest fire spreading using cellular automata and genetic algorithms , 2004, Eng. Appl. Artif. Intell..

[68]  Aníbal Ollero,et al.  Automatic Forest-Fire Measuring Using Ground Stations and Unmanned Aerial Systems , 2011, Sensors.

[69]  H. Anderson.  Fire spread and flame shape , 1968 .

[70]  Youmin Zhang,et al.  A Learning-Based Fault Tolerant Tracking Control of an Unmanned Quadrotor Helicopter , 2015, Journal of Intelligent & Robotic Systems.

[71]  Begoña C. Arrue,et al.  Laboratory fire spread analysis using visual and infrared images , 2006 .

[72]  Daniel Mandallaz,et al.  Prediction of forest fires with Poisson models , 1997 .

[73]  ByoungChul Ko,et al.  Fire-Flame Detection Based on Fuzzy Finite Automation , 2010, 2010 20th International Conference on Pattern Recognition.

[74]  Luis Merino,et al.  FIRE DETECTION USING AUTONOMOUS AERIAL VEHICLES WITH INFRARED AND VISUAL CAMERAS. , 2005 .

[75]  Feiniu Yuan,et al.  A fast accumulative motion orientation model based on integral image for video smoke detection , 2008, Pattern Recognit. Lett..

[76]  Aníbal Ollero,et al.  Cooperative Fire Detection using Unmanned Aerial Vehicles , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[77]  H.H.T. Liu,et al.  A cooperative UAV/UGV platform for wildfire detection and fighting , 2008, 2008 Asia Simulation Conference - 7th International Conference on System Simulation and Scientific Computing.

[78]  Tomàs Margalef,et al.  Enhancing wildland fire prediction on cluster systems applying evolutionary optimization techniques , 2005, Future Gener. Comput. Syst..

[79]  Vincent G. Ambrosia,et al.  Selection of Appropriate Class UAS/Sensors to Support Fire Monitoring: Experiences in the United States , 2014 .

[80]  Kerstin Wendt,et al.  Efficient knowledge retrieval to calibrate input variables in forest fire prediction , 2008 .

[81]  Agoston Restas.  Wildfire Management Supported by UAV Based Air Reconnaissance: Experiments and Results at the Szendro Fire Department, Hungary , 2006 .

[82]  Youmin Zhang,et al.  Cooperative Multi-Vehicle Search and Coverage Problem in an Uncertain Environment , 2015, Unmanned Syst..

[83]  Tzu-Hsin Kuo,et al.  Real-time video-based fire smoke detection system , 2009, 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[84]  Sung-Hwan Jung,et al.  Image Processing-Based Fire Detection System Using Statistic Color Model , 2008, 2008 International Conference on Advanced Language Processing and Web Information Technology.

[85]  Hong Jin,et al.  A fire and flame detecting method based on video , 2009, 2009 International Conference on Machine Learning and Cybernetics.

[86]  Zhang Yongming,et al.  Video Fire Smoke Detection Using Motion and Color Features , 2010 .

[87]  J.R. Martinez-De Dios,et al.  A new training-based approach for robust thresholding , 2004, Proceedings World Automation Congress, 2004..

[88]  Anh Tuan Nguyen,et al.  Using Bayes method and Fuzzy C - Mean Algorithm for Fire Detection in Video , 2009, 2009 International Conference on Advanced Technologies for Communications.

[89]  Mubarak Shah,et al.  Flame recognition in video , 2002, Pattern Recognit. Lett..

[90]  Eli Saber,et al.  Automated extraction of fire line parameters from multispectral infrared images , 2007 .

[91]  Guoqing Zhou.  Near Real-Time Orthorectification and Mosaic of Small UAV Video Flow for Time-Critical Event Response , 2009, IEEE Trans. Geosci. Remote. Sens..

[92]  Aníbal Ollero,et al.  An Intelligent System for False Alarm Reduction in Infrared Forest-Fire Detection , 2000, IEEE Intell. Syst..

[93]  Adel Hafiane,et al.  On the Evaluation of Segmentation Methods for Wildland Fire , 2009, ACIVS.

[94]  Aníbal Ollero,et al.  Journal of Intelligent & Robotic Systems manuscript No. (will be inserted by the editor) An Unmanned Aircraft System for Automatic Forest Fire Monitoring and Measurement , 2022 .

[95]  Yushan Zhu,et al.  An automatic statistical segmentation algorithm for extraction of fire and smoke regions , 2007 .

[96]  Kasim Tasdemir,et al.  Video based wildfire detection at night , 2009 .

[97]  Hong Lin,et al.  Early Warning System of Forest Fire Detection Based on Video Technology , 2014 .

[98]  Turgay Çelik,et al.  Fire detection using statistical color model in video sequences , 2007, J. Vis. Commun. Image Represent..

[99]  M. van Persie,et al.  Real-Time UAV based geospatial video integrated into the Fire Brigades crisis management GIS system , 2012 .

[100]  Chao-Ho Chen,et al.  An early fire-detection method based on image processing , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[101]  Fang Jun,et al.  Texture Analysis of Smoke for Real-Time Fire Detection , 2009, 2009 Second International Workshop on Computer Science and Engineering.

[102]  Timothy W. McLain,et al.  Cooperative forest fire surveillance using a team of small unmanned air vehicles , 2006, Int. J. Syst. Sci..

[103]  B. M. Wotton,et al.  The use of fractal dimension to improve wildland fire perimeter predictions , 1993 .

[104]  Turgay Çelik,et al.  Fire Pixel Classification using Fuzzy Logic and Statistical Color Model , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[105]  T. Chou,et al.  DISASTER MONITORING AND MANAGEMENT BY THE UNMANNED AERIAL VEHICLE TECHNOLOGY , 2010 .

[106]  Luis Merino,et al.  Multi-UAV Experiments: Application to Forest Fires , 2007 .

[107]  Amparo Alonso-Betanzos,et al.  An intelligent system for forest fire risk prediction and fire fighting management in Galicia , 2003, Expert Syst. Appl..

[108]  George Nikolakopoulos,et al.  Coordination of Helicopter UAVs for Aerial Forest-Fire Surveillance , 2009 .

[109]  Jinqiang Cui,et al.  UAV LiDAR for below-canopy forest surveys , 2013 .

[110]  Karolj Skala,et al.  Integrated System For Forest Fire Early Detection and Management , 2008 .