An Attention Enhanced Bidirectional LSTM for Early Forest Fire Smoke Recognition

Detecting forest fire smoke during the initial stages is vital for preventing forest fire events. Recent studies have shown that exploring spatial and temporal features of the image sequence is important for this task. Nevertheless, since the long distance wildfire smoke usually move slowly and lacks salient features, accurate smoke detection is still a challenging task. In this paper, we propose a novel Attention Enhanced Bidirectional Long Short-Term Memory Network (ABi-LSTM) for video based forest fire smoke recognition. The proposed ABi-LSTM consists of the spatial features extraction network, the Bidirectional Long Short-Term Memory Network(LSTM), and the temporal attention subnetwork, which can not only capture discriminative spatiotemporal features from image patch sequences but also pay different levels of attention to different patches. Experiments show that out ABi-LSTM is capable of achieving best accuracy and less false alarms on different types of scenarios. The ABi-LSTM model achieve a highly accuracy of 97.8%, and there is 4.4% improvement over the image-based deep learning model.

[1]  Wenjun Zeng,et al.  An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data , 2016, AAAI.

[2]  Lei Wang,et al.  Detection and Separation of Smoke From Single Image Frames , 2018, IEEE Transactions on Image Processing.

[3]  Feiniu Yuan,et al.  A Deep Normalization and Convolutional Neural Network for Image Smoke Detection , 2017, IEEE Access.

[4]  J. A. Ojo,et al.  Video-based Smoke Detection Algorithms: A Chronological Survey , 2014 .

[5]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Gang Li,et al.  Non-Linear Dimensionality Reduction and Gaussian Process Based Classification Method for Smoke Detection , 2017, IEEE Access.

[7]  Feiniu Yuan,et al.  A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with AdaBoost for video smoke detection , 2012, Pattern Recognit..

[8]  Yi Zhao,et al.  Saliency Detection and Deep Learning-Based Wildfire Identification in UAV Imagery , 2018, Sensors.

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Héctor M. Pérez Meana,et al.  An Early Fire Detection Algorithm Using IP Cameras , 2012, Sensors.

[12]  Qingshan Liu,et al.  Learning Multiscale Deep Features for High-Resolution Satellite Image Scene Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Steven Verstockt,et al.  Video fire detection - Review , 2013, Digit. Signal Process..

[14]  Meng Zhang,et al.  Neural Network Methods for Natural Language Processing , 2017, Computational Linguistics.

[15]  Feiniu Yuan,et al.  Video-based smoke detection with histogram sequence of LBP and LBPV pyramids , 2011 .

[16]  Shahrokh Valaee,et al.  Recent Advances in Recurrent Neural Networks , 2017, ArXiv.

[17]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[18]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Luc Van Gool,et al.  Temporal Segment Networks: Towards Good Practices for Deep Action Recognition , 2016, ECCV.

[20]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Yoshua Bengio,et al.  Attention-Based Models for Speech Recognition , 2015, NIPS.

[22]  Jing Huang,et al.  Transmission: A New Feature for Computer Vision Based Smoke Detection , 2010, AICI.

[23]  Margarita N. Favorskaya,et al.  Verification of Smoke Detection in Video Sequences Based on Spatio-temporal Local Binary Patterns , 2015, KES.

[24]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[25]  Qingshan Liu,et al.  Bidirectional-Convolutional LSTM Based Spectral-Spatial Feature Learning for Hyperspectral Image Classification , 2017, Remote. Sens..

[26]  Qixing Zhang,et al.  Adversarial Adaptation From Synthesis to Reality in Fast Detector for Smoke Detection , 2019, IEEE Access.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Qixing Zhang,et al.  Smoke Detection on Video Sequences Using 3D Convolutional Neural Networks , 2019, Fire Technology.

[29]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Thomas Brox,et al.  ECO: Efficient Convolutional Network for Online Video Understanding , 2018, ECCV.

[31]  Nikolaos Grammalidis,et al.  Higher Order Linear Dynamical Systems for Smoke Detection in Video Surveillance Applications , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Zhitao Xiao,et al.  A Dual Convolution Network Using Dark Channel Prior for Image Smoke Classification , 2019, IEEE Access.

[33]  Navdeep Jaitly,et al.  Hybrid speech recognition with Deep Bidirectional LSTM , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[34]  Lei Wang,et al.  Smoke Detection in Video: An Image Separation Approach , 2013, International Journal of Computer Vision.

[35]  Lei Wang,et al.  Single Image Smoke Detection , 2014, ACCV.

[36]  Wanqing Li,et al.  Smoke detection in videos using Non-Redundant Local Binary Pattern-based features , 2011, 2011 IEEE 13th International Workshop on Multimedia Signal Processing.