Disentangled Dynamic Deviation Transformer Networks for Multivariate Time Series Anomaly Detection

Graph neural networks have been widely used by multivariate time series-based anomaly detection algorithms to model the dependencies of system sensors. Previous studies have focused on learning the fixed dependency patterns between sensors. However, they ignore that the inter-sensor and temporal dependencies of time series are highly nonlinear and dynamic, leading to inevitable false alarms. In this paper, we propose a novel disentangled dynamic deviation transformer network (D3TN) for anomaly detection of multivariate time series, which jointly exploits multiscale dynamic inter-sensor dependencies and long-term temporal dependencies to improve the accuracy of multivariate time series prediction. Specifically, to disentangle the multiscale graph convolution, we design a novel disentangled multiscale aggregation scheme to better represent the hidden dependencies between sensors to learn fixed inter-sensor dependencies based on static topology. To capture dynamic inter-sensor dependencies determined by real-time monitoring situations and unexpected anomalies, we introduce a self-attention mechanism to model dynamic directed interactions in various potential subspaces influenced by various factors. In addition, complex temporal correlations across multiple time steps are simulated by processing the time series in parallel. Experiments on three real datasets show that the proposed D3TN significantly outperforms the state-of-the-art methods.

[1]  Muhammad Zeshan Afzal,et al.  Attention-Guided Disentangled Feature Aggregation for Video Object Detection , 2022, Sensors.

[2]  Luc Van Gool,et al.  SwinIR: Image Restoration Using Swin Transformer , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[3]  Dan Pei,et al.  Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding , 2021, KDD.

[4]  Wenwu Zhu,et al.  Multimodal Disentangled Representation for Recommendation , 2021, 2021 IEEE International Conference on Multimedia and Expo (ICME).

[5]  Anfeng Liu,et al.  BD-VTE: A Novel Baseline Data Based Verifiable Trust Evaluation Scheme for Smart Network Systems , 2021, IEEE Transactions on Network Science and Engineering.

[6]  Bryan Hooi,et al.  Graph Neural Network-Based Anomaly Detection in Multivariate Time Series , 2021, AAAI.

[7]  Kai Zheng,et al.  DAEMON: Unsupervised Anomaly Detection and Interpretation for Multivariate Time Series , 2021, 2021 IEEE 37th International Conference on Data Engineering (ICDE).

[8]  Sinno Jialin Pan,et al.  EnhanceNet: Plugin Neural Networks for Enhancing Correlated Time Series Forecasting , 2021, 2021 IEEE 37th International Conference on Data Engineering (ICDE).

[9]  Yanyan Shen,et al.  FluxEV: A Fast and Effective Unsupervised Framework for Time-Series Anomaly Detection , 2021, WSDM.

[10]  Ming Zhang,et al.  DisenHAN: Disentangled Heterogeneous Graph Attention Network for Recommendation , 2020, CIKM.

[11]  Qi Zhang,et al.  Multivariate Time-series Anomaly Detection via Graph Attention Network , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[12]  Maria A. Zuluaga,et al.  USAD: UnSupervised Anomaly Detection on Multivariate Time Series , 2020, KDD.

[13]  Chang Zhou,et al.  Disentangled Self-Supervision in Sequential Recommenders , 2020, KDD.

[14]  Xiaojun Chang,et al.  Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks , 2020, KDD.

[15]  Zhiyong Wang,et al.  Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Nicolas Loeff,et al.  Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting , 2019, International Journal of Forecasting.

[17]  Thepchai Supnithi,et al.  Detecting abnormal behavior in the transportation planning using long short term memories and a contextualized dynamic threshold , 2019, UbiComp/ISWC Adjunct.

[18]  Naixue Xiong,et al.  A Pedestrian Detection Method Based on Genetic Algorithm for Optimize XGBoost Training Parameters , 2019, IEEE Access.

[19]  Wei Sun,et al.  Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network , 2019, KDD.

[20]  Chao Yi,et al.  Time-Series Anomaly Detection Service at Microsoft , 2019, KDD.

[21]  Wei Li,et al.  Behavior sequence transformer for e-commerce recommendation in Alibaba , 2019, Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data.

[22]  Naixue Xiong,et al.  Connected Vehicle as a Mobile Sensor for Real Time Queue Length at Signalized Intersections , 2019, Sensors.

[23]  Xu Chen,et al.  Actional-Structural Graph Convolutional Networks for Skeleton-Based Action Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Lei Shi,et al.  MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks , 2019, ICANN.

[25]  Ryosuke Nakamura,et al.  Rare Event Detection Using Disentangled Representation Learning , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Naixue Xiong,et al.  Spatio-Temporal Vessel Trajectory Clustering Based on Data Mapping and Density , 2018, IEEE Access.

[27]  Naixue Xiong,et al.  An energy-efficient sleep scheduling mechanism with similarity measure for wireless sensor networks , 2018, Human-centric Computing and Information Sciences.

[28]  Valentino Constantinou,et al.  Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding , 2018, KDD.

[29]  Yang Feng,et al.  Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications , 2018, WWW.

[30]  Naixue Xiong,et al.  Exploring finger vein based personal authentication for secure IoT , 2017, Future Gener. Comput. Syst..

[31]  Charles C. Kemp,et al.  A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder , 2017, IEEE Robotics and Automation Letters.

[32]  Alexandre Termier,et al.  Anomaly Detection in Streams with Extreme Value Theory , 2017, KDD.

[33]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[34]  Naixue Xiong,et al.  A Structure Fidelity Approach for Big Data Collection in Wireless Sensor Networks , 2014, Sensors.

[35]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[36]  Piroska Haller,et al.  Data clustering-based anomaly detection in industrial control systems , 2014, 2014 IEEE 10th International Conference on Intelligent Computer Communication and Processing (ICCP).

[37]  Naixue Xiong,et al.  Adaptive GTS allocation in IEEE 802.15.4 for real-time wireless sensor networks , 2013, J. Syst. Archit..

[38]  Naixue Xiong,et al.  Privacy-preserving max/min query in two-tiered wireless sensor networks , 2013, Comput. Math. Appl..

[39]  Jiancheng Shi,et al.  The Soil Moisture Active Passive (SMAP) Mission , 2010, Proceedings of the IEEE.

[40]  Ya-Ju Fan,et al.  On the Time Series $K$-Nearest Neighbor Classification of Abnormal Brain Activity , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[41]  Francesco Battaglia,et al.  Outliers Detection in Multivariate Time Series by Independent Component Analysis , 2007, Neural Computation.

[42]  Ann B. Lee,et al.  Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[43]  J. Ma,et al.  Time-series novelty detection using one-class support vector machines , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[44]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[45]  Jianbin Huang,et al.  Detecting Urban Anomalies Using Factor Analysis and One Class Support Vector Machine , 2023, Comput. J..

[46]  Alex X. Liu,et al.  Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting , 2022, ICLR.

[47]  Pan Yang,et al.  Data Security and Privacy Protection for Cloud Storage: A Survey , 2020, IEEE Access.

[48]  Kosuke Miyoshi,et al.  Disentangled Representations for Sequence Data using Information Bottleneck Principle , 2020, ACML.

[49]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Andreas Dengel,et al.  DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series , 2019, IEEE Access.