Two-Step Real-Time Night-Time Fire Detection in an Urban Environment Using Static ELASTIC-YOLOv3 and Temporal Fire-Tube

While the number of casualties and amount of property damage caused by fires in urban areas are increasing each year, studies on their automatic detection have not maintained pace with the scale of such fire damage. Camera-based fire detection systems have numerous advantages over conventional sensor-based methods, but most research in this area has been limited to daytime use. However, night-time fire detection in urban areas is more difficult to achieve than daytime detection owing to the presence of ambient lighting such as headlights, neon signs, and streetlights. Therefore, in this study, we propose an algorithm that can quickly detect a fire at night in urban areas by reflecting its night-time characteristics. It is termed ELASTIC-YOLOv3 (which is an improvement over the existing YOLOv3) to detect fire candidate areas quickly and accurately, regardless of the size of the fire during the pre-processing stage. To reflect the dynamic characteristics of a night-time flame, N frames are accumulated to create a temporal fire-tube, and a histogram of the optical flow of the flame is extracted from the fire-tube and converted into a bag-of-features (BoF) histogram. The BoF is then applied to a random forest classifier, which achieves a fast classification and high classification performance of the tabular features to verify a fire candidate. Based on a performance comparison against a few other state-of-the-art fire detection methods, the proposed method can increase the fire detection at night compared to deep neural network (DNN)-based methods and achieves a reduced processing time without any loss in accuracy.

[1]  Turgay Çelik,et al.  Fire Pixel Classification using Fuzzy Logic and Statistical Color Model , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[2]  ByoungChul Ko,et al.  Spatiotemporal bag-of-features for early wildfire smoke detection , 2013, Image Vis. Comput..

[3]  Sung Wook Baik,et al.  Efficient Deep CNN-Based Fire Detection and Localization in Video Surveillance Applications , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[4]  Martin Mueller,et al.  Optical Flow Estimation for Flame Detection in Videos , 2013, IEEE Transactions on Image Processing.

[5]  Nikolaos Grammalidis,et al.  Spatio-Temporal Flame Modeling and Dynamic Texture Analysis for Automatic Video-Based Fire Detection , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Sercan O. Arik,et al.  TabNet: Attentive Interpretable Tabular Learning , 2019, AAAI.

[7]  Joonwhoan Lee,et al.  A Video-Based Fire Detection Using Deep Learning Models , 2019, Applied Sciences.

[8]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[9]  Sophia Karagiorgou,et al.  A service oriented architecture for decision support systems in environmental crisis management , 2012, Future Gener. Comput. Syst..

[10]  ByoungChul Ko,et al.  Fire detection based on vision sensor and support vector machines , 2009 .

[11]  ByoungChul Ko,et al.  Two-Step Cascading Algorithm for Camera-based Night Fire Detection , 2020 .

[12]  Farhat Fnaiech,et al.  Convolutional neural network for video fire and smoke detection , 2016, IECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society.

[13]  Sung Wook Baik,et al.  Convolutional Neural Networks Based Fire Detection in Surveillance Videos , 2018, IEEE Access.

[14]  Ebroul Izquierdo,et al.  A Probabilistic Approach for Vision-Based Fire Detection in Videos , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Surbhi Narwani Real-Time Fire Detection for Video Surveillance Applications Using a Combination of Experts Based On Color , Shape and Motion , 2016 .

[17]  Byoung Chul Ko,et al.  Early Detection of Sudden Pedestrian Crossing for Safe Driving During Summer Nights , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Andrew W. Senior,et al.  Long short-term memory recurrent neural network architectures for large scale acoustic modeling , 2014, INTERSPEECH.

[19]  Dong-Joong Kang,et al.  Fire detection system using random forest classification for image sequences of complex background , 2013 .

[20]  Hasan Demirel,et al.  Fire detection in video sequences using a generic color model , 2009 .

[21]  Jiaolong Xu,et al.  Deep Convolutional Neural Networks for Forest Fire Detection , 2016 .

[22]  Ali Farhadi,et al.  ELASTIC: Improving CNNs With Dynamic Scaling Policies , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Shixiao Wu,et al.  Using Popular Object Detection Methods for Real Time Forest Fire Detection , 2018, 2018 11th International Symposium on Computational Intelligence and Design (ISCID).

[24]  Mario Ignacio Chacon Murguia,et al.  Thermal Video Analysis for Fire Detection Using Shape Regularity and Intensity Saturation Features , 2011, MCPR.

[25]  Sunghwan Kim,et al.  Predictive Models of Fire via Deep learning Exploiting Colorific Variation , 2019, 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC).

[26]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  ByoungChul Ko,et al.  Modeling and Formalization of Fuzzy Finite Automata for Detection of Irregular Fire Flames , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[29]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Seong-Hoon Kim,et al.  X-ray Image Classification Using Random Forests with Local Wavelet-Based CS-Local Binary Patterns , 2011, Journal of Digital Imaging.

[31]  Toby P. Breckon,et al.  Experimentally Defined Convolutional Neural Network Architecture Variants for Non-Temporal Real-Time Fire Detection , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[32]  Chao-Ho Chen,et al.  An early fire-detection method based on image processing , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[33]  ByoungChul Ko,et al.  Early fire detection algorithm based on irregular patterns of flames and hierarchical Bayesian Networks , 2010 .

[34]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[35]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[36]  ByoungChul Ko,et al.  Nonparametric membership functions and fuzzy logic for vision sensor-based flame detection , 2010 .

[37]  A. Enis Çetin,et al.  Computer vision based method for real-time fire and flame detection , 2006, Pattern Recognit. Lett..

[38]  Sooyeong Kwak,et al.  Survey of computer vision–based natural disaster warning systems , 2012 .

[39]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[40]  ByoungChul Ko,et al.  Fire detection and 3D surface reconstruction based on stereoscopic pictures and probabilistic fuzzy logic , 2014 .

[41]  Yichao Cao,et al.  An Attention Enhanced Bidirectional LSTM for Early Forest Fire Smoke Recognition , 2019, IEEE Access.

[42]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[43]  Byoung Chul Ko,et al.  Building Deep Random Ferns Without Backpropagation , 2020, IEEE Access.

[44]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Kyriaki Kaza,et al.  Fire Detection from Images Using Faster R-CNN and Multidimensional Texture Analysis , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[46]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).