Self-adversarial Variational Autoencoder with Gaussian Anomaly Prior Distribution for Anomaly Detection

Recently, deep generative models have become increasingly popular in unsupervised anomaly detection. However, deep generative models aim at recovering the data distribution rather than detecting anomalies. Besides, deep generative models have the risk of overfitting training samples, which has disastrous effects on anomaly detection performance. To solve the above two problems, we propose a Self-adversarial Variational Autoencoder with a Gaussian anomaly prior assumption. We assume that both the anomalous and the normal prior distribution are Gaussian and have overlaps in the latent space. Therefore, a Gaussian transformer net T is trained to synthesize anomalous but near-normal latent variables. Keeping the original training objective of Variational Autoencoder, besides, the generator G tries to distinguish between the normal latent variables and the anomalous ones synthesized by T, and the encoder E is trained to discriminate whether the output of G is real. These new objectives we added not only give both G and E the ability to discriminate but also introduce additional regularization to prevent overfitting. Compared with the SOTA baselines, the proposed model achieves significant improvements in extensive experiments. Datasets and our model are available at a Github repository.

[1]  Ole Winther,et al.  Sequential Neural Models with Stochastic Layers , 2016, NIPS.

[2]  Pieter Abbeel,et al.  Variational Lossy Autoencoder , 2016, ICLR.

[3]  Noboru Harada,et al.  Complementary Set Variational Autoencoder for Supervised Anomaly Detection , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Charles C. Kemp,et al.  A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder , 2017, IEEE Robotics and Automation Letters.

[5]  Stefano Ermon,et al.  InfoVAE: Balancing Learning and Inference in Variational Autoencoders , 2019, AAAI.

[6]  Justin Bayer,et al.  Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series , 2016, ArXiv.

[7]  Charu C. Aggarwal,et al.  Theoretical Foundations and Algorithms for Outlier Ensembles , 2015, SKDD.

[8]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[9]  Yue Zhao,et al.  PyOD: A Python Toolbox for Scalable Outlier Detection , 2019, J. Mach. Learn. Res..

[10]  Fernando Nogueira,et al.  Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning , 2016, J. Mach. Learn. Res..

[11]  Hamido Fujita,et al.  Multi-Imbalance: An open-source software for multi-class imbalance learning , 2019, Knowl. Based Syst..

[12]  B. Silverman Density estimation for statistics and data analysis , 1986 .

[13]  Toby P. Breckon,et al.  GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training , 2018, ACCV.

[14]  Chenglin Wen,et al.  Deep learning fault diagnosis method based on global optimization GAN for unbalanced data , 2020, Knowl. Based Syst..

[15]  Nicu Sebe,et al.  Abnormal event detection in videos using generative adversarial nets , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[16]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[17]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[18]  Jun Li,et al.  One-Class Adversarial Nets for Fraud Detection , 2018, AAAI.

[19]  Charu C. Aggarwal,et al.  LODES: Local Density Meets Spectral Outlier Detection , 2016, SDM.

[20]  Tieniu Tan,et al.  IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis , 2018, NeurIPS.

[21]  Haibo He,et al.  A local density-based approach for outlier detection , 2017, Neurocomputing.

[22]  David J. Olive,et al.  Principal Component Analysis , 2011, International Encyclopedia of Statistical Science.

[23]  Klemens Böhm,et al.  HiCS: High Contrast Subspaces for Density-Based Outlier Ranking , 2012, 2012 IEEE 28th International Conference on Data Engineering.

[24]  Hans-Peter Kriegel,et al.  Angle-based outlier detection in high-dimensional data , 2008, KDD.

[25]  Hans-Peter Kriegel,et al.  Outlier Detection in Axis-Parallel Subspaces of High Dimensional Data , 2009, PAKDD.

[26]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[27]  Lili Yin,et al.  Active learning based support vector data description method for robust novelty detection , 2018, Knowl. Based Syst..

[28]  Navdeep Jaitly,et al.  Adversarial Autoencoders , 2015, ArXiv.

[29]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[30]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[31]  Stanislav Pidhorskyi,et al.  Generative Probabilistic Novelty Detection with Adversarial Autoencoders , 2018, NeurIPS.

[32]  Biao Huang,et al.  Process monitoring using kernel density estimation and Bayesian networking with an industrial case study. , 2015, ISA transactions.

[33]  Meng Wang,et al.  Generative Adversarial Active Learning for Unsupervised Outlier Detection , 2018, IEEE Transactions on Knowledge and Data Engineering.

[34]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[35]  Anazida Zainal,et al.  Fraud detection system: A survey , 2016, J. Netw. Comput. Appl..

[36]  Antti Honkela,et al.  Variational learning and bits-back coding: an information-theoretic view to Bayesian learning , 2004, IEEE Transactions on Neural Networks.

[37]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[38]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[39]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[40]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[41]  Asimenia Dimokranitou,et al.  Adversarial Autoencoders for Anomalous Event Detection in Images , 2017 .

[42]  Yang Feng,et al.  Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications , 2018, WWW.

[43]  Pierre Rochus,et al.  Hu, Luojia , Estimation of a censored dynamic panel data model,Econometrica. Journal of the Econometric Society , 2002 .

[44]  Seungjin Choi,et al.  Echo-state conditional variational autoencoder for anomaly detection , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[45]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[46]  Václav Smídl,et al.  Are generative deep models for novelty detection truly better? , 2018, ArXiv.

[47]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD 2000.

[48]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[49]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[50]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[51]  Joni-Kristian Kämäräinen,et al.  Gaussian mixture pdf in one-class classification: computing and utilizing confidence values , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[52]  Leman Akoglu,et al.  Less is More , 2016, ACM Trans. Knowl. Discov. Data.

[53]  Yupu Yang,et al.  Improved nonlinear process monitoring based on ensemble KPCA with local structure analysis , 2019, Chemical Engineering Research and Design.

[54]  Haibo He,et al.  ADASYN: Adaptive synthetic sampling approach for imbalanced learning , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[55]  Artur Gramacki,et al.  FFT-based fast bandwidth selector for multivariate kernel density estimation , 2015, Comput. Stat. Data Anal..

[56]  Andreas Dengel,et al.  Histogram-based Outlier Score (HBOS): A fast Unsupervised Anomaly Detection Algorithm , 2012 .

[57]  Farid Kadri,et al.  Improved principal component analysis for anomaly detection: Application to an emergency department , 2015, Comput. Ind. Eng..

[58]  Takashi Nishide,et al.  Network Intrusion Detection Based on Semi-supervised Variational Auto-Encoder , 2017, ESORICS.

[59]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.

[60]  Hazem N. Nounou,et al.  Iterated Robust kernel Fuzzy Principal Component Analysis and application to fault detection , 2016, J. Comput. Sci..

[61]  Dit-Yan Yeung,et al.  Parzen-window network intrusion detectors , 2002, Object recognition supported by user interaction for service robots.

[62]  Raghavendra Chalapathy University of Sydney,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.