Robust unsupervised anomaly detection via multi-time scale DCGANs with forgetting mechanism for industrial multivariate time series

Abstract Detecting anomalies in time series is a vital technique in a wide variety of industrial application in which sensors monitor expensive machinery. The complexity of this task increases when heterogeneous sensors provide information of different attributes, scales and characteristics from the same machine. Actually, the challenges of anomaly detection for industrial time series are to design effective pre-processing, feature extraction and overcome the lack of abnormal samples. Recent deep learning models have shown prominent abilities on raw multivariate time series, alleviating these previous works. In this work, a novel framework named multi-time scale deep convolutional generative adversarial network (MTS-DCGAN) is proposed to deal with anomaly detection of industrial time series. Firstly, multivariate time series are transformed into the multi-channel signature matrices via sliding window based cross-correlation computation, and therein forgetting mechanism is introduced to effectively avoid false alarms due to excessive influence of old sequences. Then the framework conducts an unsupervised adversarial training of multi-channel signature matrices and capture their hidden features via deep convolution structure. Besides, a novel threshold setting strategy is proposed to optimize anomaly detection performance under imbalance of normal and abnormal data. Finally, the proposed framework is assessed against the experiments on four datasets. Within this study, experimental results show our framework outperforms the comparison algorithms in terms of model performance and robustness, providing an effective anomaly detection method for industrial multivariate time series.

[1]  Ejaz Ahmed,et al.  Real-time big data processing for anomaly detection: A Survey , 2019, Int. J. Inf. Manag..

[2]  Yi Wang,et al.  A deep learning approach for anomaly detection based on SAE and LSTM in mechanical equipment , 2019, The International Journal of Advanced Manufacturing Technology.

[3]  Jack Beuth,et al.  Anomaly Detection and Classification in a Laser Powder Bed Additive Manufacturing Process using a Trained Computer Vision Algorithm , 2018 .

[4]  Jie Wang,et al.  The Application of a Double CUSUM Algorithm in Industrial Data Stream Anomaly Detection , 2018, Symmetry.

[5]  Fei Tony Liu,et al.  Isolation-Based Anomaly Detection , 2012, TKDD.

[6]  Francesco Carlo Morabito,et al.  A novel statistical analysis and autoencoder driven intelligent intrusion detection approach , 2020, Neurocomputing.

[7]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[8]  Miriam A. M. Capretz,et al.  An ensemble learning framework for anomaly detection in building energy consumption , 2017 .

[9]  Brian W. Baetz,et al.  A random forest model for inflow prediction at wastewater treatment plants , 2019, Stochastic Environmental Research and Risk Assessment.

[10]  Michele Luvisotto,et al.  Distributed Clustering Strategies in Industrial Wireless Sensor Networks , 2017, IEEE Transactions on Industrial Informatics.

[11]  Evgin Goceri,et al.  Analysis of Deep Networks with Residual Blocks and Different Activation Functions: Classification of Skin Diseases , 2019, 2019 Ninth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[12]  Gurjit Singh Walia,et al.  Crowd anomaly detection using Aggregation of Ensembles of fine-tuned ConvNets , 2020, Neurocomputing.

[13]  Georg Langs,et al.  f‐AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks , 2019, Medical Image Anal..

[14]  Lei Shi,et al.  MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks , 2019, ICANN.

[15]  Mohamed Cheriet,et al.  Multiple-Step-Ahead Traffic Prediction in High-Speed Networks , 2018, IEEE Communications Letters.

[16]  Suleyman Serdar Kozat,et al.  Unsupervised Anomaly Detection With LSTM Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Ling Lin,et al.  Anomaly detection method for sensor network data streams based on sliding window sampling and optimized clustering , 2019 .

[18]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[19]  B. Matthews Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[20]  Rajeev Kumar,et al.  Receiver operating characteristic (ROC) curve for medical researchers , 2011, Indian pediatrics.

[21]  Andreas Theissler,et al.  Detecting known and unknown faults in automotive systems using ensemble-based anomaly detection , 2017, Knowl. Based Syst..

[22]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[23]  Huixin Zhou,et al.  Hyperspectral anomaly detection by local joint subspace process and support vector machine , 2020, International Journal of Remote Sensing.

[24]  Francisco Herrera,et al.  Transforming big data into smart data: An insight on the use of the k‐nearest neighbors algorithm to obtain quality data , 2018, WIREs Data Mining Knowl. Discov..

[25]  Evgin Goceri,et al.  Diagnosis of Alzheimer's disease with Sobolev gradient‐based optimization and 3D convolutional neural network , 2019, International journal for numerical methods in biomedical engineering.

[26]  Mohsen Guizani,et al.  Internet of Things Architecture: Recent Advances, Taxonomy, Requirements, and Open Challenges , 2017, IEEE Wireless Communications.

[27]  Bizhong Xia,et al.  Unsupervised Anomaly Detection of Industrial Robots Using Sliding-Window Convolutional Variational Autoencoder , 2020, IEEE Access.

[28]  Yiannis S. Boutalis,et al.  Exploiting the generative adversarial framework for one-class multi-dimensional fault detection , 2019, Neurocomputing.

[29]  Tianrui Li,et al.  Multivariate time series forecasting via attention-based encoder-decoder framework , 2020, Neurocomputing.

[30]  Guang Li,et al.  Application of Sliding Nest Window Control Chart in Data Stream Anomaly Detection , 2018, Symmetry.

[31]  Xin Huang,et al.  Robust and Rapid Adaption for Concept Drift in Software System Anomaly Detection , 2018, 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE).

[32]  Luca Benini,et al.  A semisupervised autoencoder-based approach for anomaly detection in high performance computing systems , 2019, Eng. Appl. Artif. Intell..

[33]  Chao Liu,et al.  An unsupervised spatiotemporal graphical modeling approach for wind turbine condition monitoring , 2018, Renewable Energy.

[34]  Ning Xia,et al.  Deep r -th Root of Rank Supervised Joint Binary Embedding for Multivariate Time Series Retrieval , 2018, KDD.

[35]  Seiichi Uchida,et al.  A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data , 2016, PloS one.

[36]  Daniela Schulz,et al.  The sliding window correlation procedure for detecting hidden correlations: existence of behavioral subgroups illustrated with aged rats , 2002, Journal of Neuroscience Methods.

[37]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[38]  Zhou Yang,et al.  Robust deep auto-encoding Gaussian process regression for unsupervised anomaly detection , 2020, Neurocomputing.

[39]  Nanjun Li,et al.  Video anomaly detection and localization via multivariate gaussian fully convolution adversarial autoencoder , 2019, Neurocomputing.

[40]  Ming Xu,et al.  Anomaly Detection in Road Networks Using Sliding-Window Tensor Factorization , 2018, IEEE Transactions on Intelligent Transportation Systems.

[41]  Ying Wei,et al.  Data-driven bearing fault identification using improved hidden Markov model and self-organizing map , 2018, Comput. Ind. Eng..

[42]  Vishal M. Patel,et al.  Learning Deep Features for One-Class Classification , 2018, IEEE Transactions on Image Processing.

[43]  Hashem M. Hashemian,et al.  State-of-the-Art Predictive Maintenance Techniques* , 2011, IEEE Transactions on Instrumentation and Measurement.

[44]  Enrique Onieva,et al.  Multi-head CNN-RNN for multi-time series anomaly detection: An industrial case study , 2019, Neurocomputing.

[45]  K. Hajian‐Tilaki,et al.  Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation. , 2013, Caspian journal of internal medicine.

[46]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[47]  Benjamin Lindemann,et al.  Detektion von Anomalien zur Qualitätssicherung basierend auf Sequence-to-Sequence LSTM Netzen , 2019, Autom..

[48]  Liang Zhang,et al.  A weakly supervised framework for abnormal behavior detection and localization in crowded scenes , 2020, Neurocomputing.

[49]  Yang Lyu,et al.  A Generic Anomaly Detection of Catenary Support Components Based on Generative Adversarial Networks , 2020, IEEE Transactions on Instrumentation and Measurement.

[50]  Francisco Herrera,et al.  Learning from Imbalanced Data Sets , 2018, Springer International Publishing.