Robust Subspace Recovery Layer for Unsupervised Anomaly Detection

We propose a neural network for unsupervised anomaly detection with a novel robust subspace recovery layer (RSR layer). This layer seeks to extract the underlying subspace from a latent representation of the given data and removes outliers that lie away from this subspace. It is used within an autoencoder. The encoder maps the data into a latent space, from which the RSR layer extracts the subspace. The decoder then smoothly maps back the underlying subspace to a "manifold" close to the original inliers. Inliers and outliers are distinguished according to the distances between the original and mapped positions (small for inliers and large for outliers). Extensive numerical experiments with both image and document datasets demonstrate state-of-the-art precision and recall.

[1]  Anand Rajaraman,et al.  Mining of Massive Datasets , 2011 .

[2]  Peter W. Jones Rectifiable sets and the Traveling Salesman Problem , 1990 .

[3]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[4]  Fei Tony Liu,et al.  Isolation-Based Anomaly Detection , 2012, TKDD.

[5]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[6]  Gilad Lerman,et al.  Median K-Flats for hybrid linear modeling with many outliers , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[7]  Sanjay Chawla,et al.  Robust, Deep and Inductive Anomaly Detection , 2017, ECML/PKDD.

[8]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[9]  Seiichi Uchida,et al.  A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data , 2016, PloS one.

[10]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[11]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[12]  Sungzoon Cho,et al.  Variational Autoencoder based Anomaly Detection using Reconstruction Probability , 2015 .

[13]  Gilad Lerman,et al.  A Well-Tempered Landscape for Non-convex Robust Subspace Recovery , 2017, J. Mach. Learn. Res..

[14]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[15]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[16]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[17]  Randy C. Paffenroth,et al.  Robust PCA for Anomaly Detection in Cyber Networks , 2018, ArXiv.

[18]  En Zhu,et al.  Deep Clustering with Convolutional Autoencoders , 2017, ICONIP.

[19]  G. Lerman Quantifying curvelike structures of measures by using L2 Jones quantities , 2003 .

[20]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[21]  Gang Hua,et al.  Learning Discriminative Reconstructions for Unsupervised Outlier Removal , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[23]  Guillermo Sapiro,et al.  OLE: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Namrata Vaswani,et al.  Static and Dynamic Robust PCA and Matrix Completion: A Review , 2018, Proceedings of the IEEE.

[25]  A. G. Some problems in orthogonal distance and non-orthogonal distance regression , 2001 .

[26]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[27]  Chuan Sheng Foo,et al.  Efficient GAN-Based Anomaly Detection , 2018, ArXiv.

[28]  Gilad Lerman,et al.  $${l_p}$$lp-Recovery of the Most Significant Subspace Among Multiple Subspaces with Outliers , 2010, ArXiv.

[29]  J. Tropp,et al.  Two proposals for robust PCA using semidefinite programming , 2010, 1012.1086.

[30]  Shenghua Gao,et al.  Future Frame Prediction for Anomaly Detection - A New Baseline , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[32]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[33]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[34]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[35]  Slim Abdennadher,et al.  Enhancing one-class support vector machines for unsupervised anomaly detection , 2013, ODD '13.

[36]  Gilad Lerman,et al.  An Overview of Robust Subspace Recovery , 2018, Proceedings of the IEEE.

[37]  S. Semmes,et al.  Analysis of and on uniformly rectifiable sets , 1993 .

[38]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[39]  Daniel Cremers,et al.  q-Space Novelty Detection with Variational Autoencoders , 2018, Computational Diffusion MRI.

[40]  Ran El-Yaniv,et al.  Deep Anomaly Detection Using Geometric Transformations , 2018, NeurIPS.

[41]  Joel A. Tropp,et al.  Robust Computation of Linear Models by Convex Relaxation , 2012, Foundations of Computational Mathematics.

[42]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[43]  David D. Lewis,et al.  Reuters-21578 Text Categorization Test Collection, Distribution 1.0 , 1997 .

[44]  Gilad Lerman,et al.  A novel M-estimator for robust PCA , 2011, J. Mach. Learn. Res..

[45]  Yu Cheng,et al.  Deep Structured Energy Based Models for Anomaly Detection , 2016, ICML.

[46]  Shachar Fleishman,et al.  Novelty Detection with GAN , 2018, ArXiv.

[47]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[48]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[49]  Francesco Cricri,et al.  Clustering and Unsupervised Anomaly Detection with l2 Normalized Deep Auto-Encoder Representations , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[50]  M. Shyu,et al.  A Novel Anomaly Detection Scheme Based on Principal Component Classifier , 2003 .

[51]  Gilad Lerman,et al.  Robust Subspace Recovery with Adversarial Outliers , 2019, ArXiv.

[52]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[53]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[54]  Gilad Lerman,et al.  Fast, Robust and Non-convex Subspace Recovery , 2014, 1406.6145.

[55]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[56]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[57]  Charu C. Aggarwal,et al.  Outlier Detection for Text Data , 2017, SDM.