Light-Weight Student LSTM for Real-Time Wildfire Smoke Detection

As the need for wildfire detection increases, research on wildfire smoke detection combining low-cost cameras and deep learning technology is increasing. Camera-based wildfire smoke detection is inexpensive, allowing for a quick detection, and allows a smoke to be checked by the naked eye. However, because a surveillance system must rely only on visual characteristics, it often erroneously detects fog and clouds as smoke. In this study, a combination of a You-Only-Look-Once detector and a long short-term memory (LSTM) classifier is applied to improve the performance of wildfire smoke detection by reflecting on the spatial and temporal characteristics of wildfire smoke. However, because it is necessary to lighten the heavy LSTM model for real-time smoke detection, in this paper, we propose a new method for applying the teacher–student framework to deep LSTM. Through this method, a shallow student LSTM is designed to reduce the number of layers and cells constituting the LSTM model while maintaining the original deep LSTM performance. As the experimental results indicate, our proposed method achieves up to an 8.4-fold decrease in the number of parameters and a faster processing time than the teacher LSTM while maintaining a similar detection performance as deep LSTM using several state-of-the-art methods on a wildfire benchmark dataset.

[1]  Shuo Zhang,et al.  Wildfire Detection Using Sound Spectrum Analysis Based on the Internet of Things , 2019, Sensors.

[2]  Sooyeong Kwak,et al.  Survey of computer vision–based natural disaster warning systems , 2012 .

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Yuan-Fang Wang,et al.  Smoke Detection in Video , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[5]  Triston Blalack,et al.  Low-Power Distributed Sensor Network for Wildfire Detection , 2019, 2019 SoutheastCon.

[6]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[7]  Robert S. Allison,et al.  Airborne Optical and Thermal Remote Sensing for Wildfire Detection and Monitoring , 2016, Sensors.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Chao-Ho Chen,et al.  The smoke detection for early fire-alarming system base on video processing , 2006, 2006 International Conference on Intelligent Information Hiding and Multimedia.

[10]  Yanjun Li,et al.  Wireless Sensor Network Design for Wildfire Monitoring , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[11]  Joonwhoan Lee,et al.  A Video-Based Fire Detection Using Deep Learning Models , 2019, Applied Sciences.

[12]  ByoungChul Ko,et al.  Spatiotemporal bag-of-features for early wildfire smoke detection , 2013, Image Vis. Comput..

[13]  A. Enis Çetin,et al.  Early Wildfire Smoke Detection Based on Motion-based Geometric Image Transformation and Deep Convolutional Generative Adversarial Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  A. Enis Çetin,et al.  Wavelet based real-time smoke detection in video , 2005, 2005 13th European Signal Processing Conference.

[15]  Zhang Yongming,et al.  Video Fire Smoke Detection Using Motion and Color Features , 2010 .

[16]  Yun Teng,et al.  CornerNet-Lite: Efficient Keypoint based Object Detection , 2019, BMVC.

[17]  Feiniu Yuan,et al.  Video-based smoke detection with histogram sequence of LBP and LBPV pyramids , 2011 .

[18]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Qixing Zhang,et al.  Smoke Detection on Video Sequences Using 3D Convolutional Neural Networks , 2019, Fire Technology.

[20]  Yi Zhao,et al.  Saliency Detection and Deep Learning-Based Wildfire Identification in UAV Imagery , 2018, Sensors.

[21]  Marimuthu Palaniswami,et al.  Smoke detection in video using wavelets and support vector machines , 2009 .

[22]  Sung Wook Baik,et al.  Energy-Efficient Deep CNN for Smoke Detection in Foggy IoT Environment , 2019, IEEE Internet of Things Journal.

[23]  Qi Tian,et al.  CenterNet: Keypoint Triplets for Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Latha Parameswaran,et al.  LSTM and GRU Deep Learning Architectures for Smoke Prediction System in Indoor Environment , 2021 .

[26]  Pan Wang,et al.  Smoke Detection Based on Deep Convolutional Neural Networks , 2016, 2016 International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Industrial Information Integration (ICIICII).

[27]  Xiaohui Xie,et al.  Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks , 2016, AAAI.

[28]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[29]  Qixing Zhang,et al.  Wildland Forest Fire Smoke Detection Based on Faster R-CNN using Synthetic Smoke Images , 2018 .

[30]  ByoungChul Ko,et al.  Wildfire smoke detection using temporospatial features and random forest classifiers , 2012 .

[31]  Sangjun Kim,et al.  Fast Pedestrian Detection in Surveillance Video Based on Soft Target Training of Shallow Random Forest , 2019, IEEE Access.

[32]  Springer Fachmedien Wiesbaden,et al.  Service , 2018, Wirtschaftsinformatik Manag..

[33]  Tian Han,et al.  Characterizing boreal forest wildfire with multi-temporal Landsat and LIDAR data , 2009 .

[34]  Baijian Yang,et al.  Deep Learning Based Wildfire Event Object Detection from 4K Aerial Images Acquired by UAS , 2020, AI.

[35]  Kang-Hyun Jo,et al.  Smoke Detection on Video Sequences Using Convolutional and Recurrent Neural Networks , 2017, ICCCI.

[36]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[38]  Byoung Chul Ko,et al.  Two-Step Real-Time Night-Time Fire Detection in an Urban Environment Using Static ELASTIC-YOLOv3 and Temporal Fire-Tube , 2020, Sensors.

[39]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[40]  Gao Xu,et al.  Video Smoke Detection Based on Deep Saliency Network , 2018, Fire Safety Journal.

[41]  Michael I. Jordan Serial Order: A Parallel Distributed Processing Approach , 1997 .

[42]  Feiniu Yuan,et al.  A Deep Normalization and Convolutional Neural Network for Image Smoke Detection , 2017, IEEE Access.

[43]  Farhat Fnaiech,et al.  Convolutional neural network for video fire and smoke detection , 2016, IECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society.

[44]  Koichi Shinoda,et al.  Wise teachers train better DNN acoustic models , 2016, EURASIP J. Audio Speech Music. Process..

[45]  Xiaoqiao Meng,et al.  Real-time forest fire detection with wireless sensor networks , 2005, Proceedings. 2005 International Conference on Wireless Communications, Networking and Mobile Computing, 2005..

[46]  Yichao Cao,et al.  An Attention Enhanced Bidirectional LSTM for Early Forest Fire Smoke Recognition , 2019, IEEE Access.

[47]  Jiaolong Xu,et al.  Deep Convolutional Neural Networks for Forest Fire Detection , 2016 .

[48]  Yi He,et al.  Deep LSTM for Large Vocabulary Continuous Speech Recognition , 2017, ArXiv.