Deep anomaly detection for industrial systems: a case study

We explore the use of deep neural networks for anomaly detection of industrial systems where the data are multivariate time series measurements. We formulate the problem as a self-supervised learning where data under normal operation is used to train a deep neural network autoregressive model, i.e., use a window of time series data to predict future data values. The aim of such a model is to learn to represent the system dynamic behavior under normal conditions, while expect higher model vs. measurement discrepancies under faulty conditions. In real world applications, many control settings are categorical in nature. In this paper, vector embedding and joint losses are employed to deal with such situations. Both LSTM and CNN based deep neural network backbones are studied on the Secure Water Treatment (SWaT) testbed datasets. Also, Support Vector Data Description (SVDD) method is adapted to such anomaly detection settings with deep neural networks. Evaluation methods and results are discussed based on the SWaT dataset along with potential pitfalls.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Fenglong Ma,et al.  MuVAN: A Multi-view Attention Network for Multivariate Temporal Data , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[3]  Yifan Guo,et al.  Multidimensional Time Series Anomaly Detection: A GRU-based Gaussian Mixture Variational Autoencoder Approach , 2018, ACML.

[4]  Hyunki Lim,et al.  GAN-Based Anomaly Detection and Localization of Multivariate Time Series Data for Power Plant , 2020, 2020 IEEE International Conference on Big Data and Smart Computing (BigComp).

[5]  Andreas Dengel,et al.  DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series , 2019, IEEE Access.

[6]  Sridhar Adepu,et al.  A Dataset to Support Research in the Design of Secure Water Treatment Systems , 2016, CRITIS.

[7]  Tailai Wen,et al.  Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning , 2019, ArXiv.

[8]  Wan-Lei Zhao,et al.  Sequential VAE-LSTM for Anomaly Detection on Time Series , 2019, ArXiv.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Sanjay Chawla,et al.  Anomaly Detection using One-Class Neural Networks , 2018, ArXiv.

[11]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[12]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[13]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[14]  Weizhong Yan,et al.  Detecting Gas Turbine Combustor Anomalies Using Semi-supervised Anomaly Detection with Deep Representation Learning , 2019, Cognitive Computation.

[15]  Lei Shi,et al.  MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks , 2019, ICANN.

[16]  Jun Sun,et al.  Anomaly Detection for a Water Treatment System Using Unsupervised Machine Learning , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[17]  Raghavendra Chalapathy University of Sydney,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.

[18]  Subutai Ahmad,et al.  Unsupervised real-time anomaly detection for streaming data , 2017, Neurocomputing.

[19]  Sungzoon Cho,et al.  Variational Autoencoder based Anomaly Detection using Reconstruction Probability , 2015 .

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[23]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[24]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[25]  Suleyman Serdar Kozat,et al.  Unsupervised Anomaly Detection With LSTM Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Takehisa Yairi,et al.  A review on the application of deep learning in system health management , 2018, Mechanical Systems and Signal Processing.

[27]  Hans-Peter Kriegel,et al.  A survey on unsupervised outlier detection in high‐dimensional numerical data , 2012, Stat. Anal. Data Min..

[28]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.