TS-Bert: Time Series Anomaly Detection via Pre-training Model Bert

Anomaly detection of time series is of great importance in data mining research. Current state of the art suffer from scalability, over reliance on labels and high false positives. To this end, a novel framework, named TS-Bert, is proposed in this paper. TS-Bert is based on pre-training model Bert and consists of two phases, accordingly. In the pre-training phase, the model learns the behavior features of the time series from massive unlabeled data. In the fine-tuning phase, the model is fine-tuned based on the target dataset. Since the Bert model is not designed for the time series anomaly detection task, we have made some modifications thus to improve the detection accuracy. Furthermore, we have removed the dependency of the model on labeled data so that TS-Bert is unsupervised. Experiments on the public data set KPI and yahoo demonstrate that TS-Bert has significantly improved the f1 value compared to the current state-of-the-art unsupervised learning models.

[1]  Dominique T. Shipmon,et al.  Time Series Anomaly Detection; Detection of anomalous drops with limited features and sparse examples in noisy highly periodic data , 2017, ArXiv.

[2]  Reda Alhajj,et al.  Fourier Transform Based Spatial Outlier Mining , 2009, IDEAL.

[3]  D. Dickey,et al.  Testing for unit roots in autoregressive-moving average models of unknown order , 1984 .

[4]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[5]  Chris Chatfield,et al.  The Holt-Winters Forecasting Procedure , 1978 .

[6]  Yang Feng,et al.  Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications , 2018, WWW.

[7]  Albert G. Greenberg,et al.  Network anomography , 2005, IMC '05.

[8]  Bo Zong,et al.  A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data , 2018, AAAI.

[9]  Xing Wu,et al.  Conditional BERT Contextual Augmentation , 2018, ICCS.

[10]  Marius Kloft,et al.  Toward Supervised Anomaly Detection , 2014, J. Artif. Intell. Res..

[11]  Chao Yi,et al.  Time-Series Anomaly Detection Service at Microsoft , 2019, KDD.

[12]  Jimmy J. Lin,et al.  Simple Applications of BERT for Ad Hoc Document Retrieval , 2019, ArXiv.

[13]  Jesse Vig,et al.  Comparison of Transfer-Learning Approaches for Response Selection in Multi-Turn Conversations , 2018 .

[14]  Saeed Amizadeh,et al.  Generic and Scalable Framework for Automated Time-series Anomaly Detection , 2015, KDD.

[15]  Lovekesh Vig,et al.  LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection , 2016, ArXiv.

[16]  Wen Wang,et al.  BERT for Joint Intent Classification and Slot Filling , 2019, ArXiv.

[17]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[18]  Enrique Onieva,et al.  Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study , 2019, Neurocomputing.

[19]  Ali A. Ghorbani,et al.  Network Anomaly Detection Based on Wavelet Analysis , 2009, EURASIP J. Adv. Signal Process..

[20]  Dan Pei,et al.  Opprentice: Towards Practical and Automatic Anomaly Detection Through Machine Learning , 2015, Internet Measurement Conference.