Multi-Head Attention based Bi-LSTM for Anomaly Detection in Multivariate Time-Series of WSN

Anomaly detection is a widely utilized technique in the field of wireless sensor networks (WSNs) data stream analysis, aimed at identifying unusual events or anomalies in an early stage. However, the constraints of WSN applications pose a significant challenge in achieving effective and efficient anomaly detection. In this work, we proposes a new multi-head attention-based Bi-LSTM approach for anomaly detection in multivariate time-series. Rather than modeling the time series of individual sensor independently, the proposed approach models the time series of multiple sensors concurrently, taking into account potential latent interactions among them, thereby enhancing the accuracy of anomaly detection. The proposed approach does not require labeled data and can be directly applied in real-world scenarios where labeling a large stream of data from heterogeneous sensors is both difficult and time-consuming. Finally, empirical evaluations using a real-world WSN demonstrate effectiveness and robustness of the proposed approach, outperforming traditional deep learning approaches.

[1]  S. Wshah,et al.  Transformer-based deep learning model for forced oscillation localization , 2023, International Journal of Electrical Power & Energy Systems.

[2]  Qinghao Zhang,et al.  A novel anomaly detection method for multimodal WSN data flow via a dynamic graph neural network , 2022, Connect. Sci..

[3]  Junchi Yan,et al.  Transformers in Time Series: A Survey , 2022, IJCAI.

[4]  Xiuzhen Cheng,et al.  Learning Graph Structures With Transformer for Multivariate Time-Series Anomaly Detection in IoT , 2021, IEEE Internet of Things Journal.

[5]  Xingsi Xue,et al.  Integrating Sensor Ontologies with Global and Local Alignment Extractions , 2021, Wirel. Commun. Mob. Comput..

[6]  Wencheng Wu,et al.  Deep Learning for Model Parameter Calibration in Power Systems , 2020, 2020 IEEE International Conference on Power Systems Technology (POWERCON).

[7]  B. Paramasivan,et al.  Anomaly detection in wireless sensor network using machine learning algorithm , 2020, Comput. Commun..

[8]  Margarida Silveira,et al.  Unsupervised Anomaly Detection in Energy Time Series Data Using Variational Recurrent Autoencoders with Attention , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[9]  Fiza Abdul Rahim,et al.  A Survey on Anomalies Detection Techniques and Measurement Methods , 2018, 2018 IEEE Conference on Application, Information and Network Security (AINS).

[10]  Ye Yuan,et al.  A Comparative Analysis of SVM, Naive Bayes and GBDT for Data Faults Detection in WSNs , 2018, 2018 IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C).

[11]  Yang Xu,et al.  Application of Wireless Sensor Network in Water Quality Monitoring , 2017, 22017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC).

[12]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[13]  Lovekesh Vig,et al.  LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection , 2016, ArXiv.

[14]  Charles Elkan,et al.  Learning to Diagnose with LSTM Recurrent Neural Networks , 2015, ICLR.

[15]  Erik Marchi,et al.  A novel approach for automatic acoustic novelty detection using a denoising autoencoder with bidirectional LSTM neural networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Carla E. Brodley,et al.  FRaC: a feature-modeling approach for semi-supervised and unsupervised anomaly detection , 2012, Data Mining and Knowledge Discovery.

[18]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[19]  Giovanni Soda,et al.  Exploiting the past and the future in protein secondary structure prediction , 1999, Bioinform..

[20]  Rainer Hoch,et al.  On the evaluation of document analysis components by recall, precision, and accuracy , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[21]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[22]  Miao Xie,et al.  Anomaly Detection in Wireless Sensor Networks , 2013 .