Time Series Encodings with Temporal Convolutional Networks

The training of anomaly detection models usually requires labeled data. We present in this paper a novel approach for anomaly detection in time series which trains unsupervised using a convolutional approach coupled to an autoencoder framework. After training, only a small amount of labeled data is needed to adjust the anomaly threshold. We show that our new approach outperforms several other state-of-the-art anomaly detection algorithms on a Mackey-Glass (MG) anomaly benchmark. At the same time our autoencoder is capable of learning interesting representations in latent space. Our new MG anomaly benchmark allows to create an unlimited amount of anomaly benchmark data with steerable difficulty. In this benchmark, the anomalies are well-defined, yet difficult to spot for the human eye.

[1]  Andreas Dengel,et al.  DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series , 2019, IEEE Access.

[2]  John Canny,et al.  GPU accelerated t-distributed stochastic neighbor embedding , 2019, J. Parallel Distributed Comput..

[3]  Subutai Ahmad,et al.  Evaluating Real-Time Anomaly Detection Algorithms -- The Numenta Anomaly Benchmark , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[4]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[5]  Yann Dauphin,et al.  Language Modeling with Gated Convolutional Networks , 2016, ICML.

[6]  Justin Bayer,et al.  Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series , 2016, ArXiv.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[9]  Chris Eliasmith,et al.  Hyperopt: a Python library for model selection and hyperparameter optimization , 2015 .

[10]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[11]  Margarida Silveira,et al.  Unsupervised Anomaly Detection in Energy Time Series Data Using Variational Recurrent Autoencoders with Attention , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[12]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[13]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[14]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[15]  Yang Feng,et al.  Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications , 2018, WWW.

[16]  Lovekesh Vig,et al.  LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection , 2016, ArXiv.

[17]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[18]  Thomas Bäck,et al.  Anomaly Detection in Electrocardiogram Readings with Stacked LSTM Networks , 2019, ITAT.

[19]  Gerrit Ansmann,et al.  Efficiently and easily integrating differential equations with JiTCODE, JiTCDDE, and JiTCSDE , 2017, Chaos.

[20]  Lei Shi,et al.  MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks , 2019, ICANN.

[21]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[22]  Hongxing He,et al.  Outlier Detection Using Replicator Neural Networks , 2002, DaWaK.

[23]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[24]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[25]  Yang Hong,et al.  A GAN-Based Anomaly Detection Approach for Imbalanced Industrial Time Series , 2019, IEEE Access.

[26]  Yann Dauphin,et al.  Convolutional Sequence to Sequence Learning , 2017, ICML.

[27]  Thomas Bäck,et al.  Online anomaly detection on the webscope S5 dataset: A comparative study , 2017, 2017 Evolving and Adaptive Intelligent Systems (EAIS).

[28]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .