Deep Random Projection Outlyingness for Unsupervised Anomaly Detection

Random projection is a common technique for designing algorithms in a variety of areas, including information retrieval, compressive sensing and measuring of outlyingness. In this work, the original random projection outlyingness measure is modified and associated with a neural network to obtain an unsupervised anomaly detection method able to handle multimodal normality. Theoretical and experimental arguments are presented to justify the choices of the anomaly score estimator, the dimensions of the random projections, and the number of such projections. The contribution of adapted dropouts is investigated, along with the affine stability of the proposed method. The performance of the proposed neural network approach is comparable to a state-of-the-art anomaly detection method. Experiments conducted on the MNIST, Fashion-MNIST and CIFAR-10 datasets show the relevance of the proposed approach, and suggest a possible extension to a semi-supervised setup.

[1]  Marcin Kurdziel,et al.  Training neural networks on high-dimensional data using random projection , 2018, Pattern Analysis and Applications.

[2]  D. Donoho,et al.  Breakdown Properties of Location Estimates Based on Halfspace Depth and Projected Outlyingness , 1992 .

[3]  Pierre Soille,et al.  Local Mutual Information for Dissimilarity-Based Image Segmentation , 2014, Journal of Mathematical Imaging and Vision.

[4]  Zhenghao Chen,et al.  On Random Weights and Unsupervised Feature Learning , 2011, ICML.

[5]  Santiago Velasco-Forero,et al.  From unsupervised to semi-supervised anomaly detection methods for HRRP targets , 2020, 2020 IEEE Radar Conference (RadarConf20).

[6]  Heikki Mannila,et al.  Random projection in dimensionality reduction: applications to image and text data , 2001, KDD '01.

[7]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[8]  Raghavendra Chalapathy University of Sydney,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.

[9]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Alexander Binder,et al.  Deep Semi-Supervised Anomaly Detection , 2019, ICLR.

[12]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[13]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[14]  Qian Du,et al.  Anomaly Detection and Reconstruction From Random Projections , 2012, IEEE Transactions on Image Processing.

[15]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[16]  David J. Schwab,et al.  Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs , 2020, ICLR.

[17]  Y. Zuo Projection-based depth functions and associated medians , 2003 .

[18]  AI Koan,et al.  Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning , 2008, NIPS.

[19]  Thomas G. Dietterich,et al.  A Unifying Review of Deep and Shallow Anomaly Detection , 2020, Proceedings of the IEEE.

[20]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[21]  Thomas G. Dietterich,et al.  Deep Anomaly Detection with Outlier Exposure , 2018, ICLR.

[22]  Guillermo Sapiro,et al.  Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy? , 2015, IEEE Transactions on Signal Processing.

[23]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[24]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[25]  P. Rousseeuw,et al.  Minimum volume ellipsoid , 2009 .

[26]  Jesús Angulo,et al.  Random Projection Depth for Multivariate Mathematical Morphology , 2012, IEEE Journal of Selected Topics in Signal Processing.

[27]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[28]  Jesús Angulo,et al.  Robust RX anomaly detector without covariance matrix estimation , 2012, 2012 4th Workshop on Hyperspectral Image and Signal Processing (WHISPERS).

[29]  Rethinking Assumptions in Deep Anomaly Detection , 2020, ArXiv.

[30]  Kim Phuc Tran,et al.  Anomaly detection in wireless sensor networks via support vector data description with mahalanobis kernels and discriminative adjustment , 2017, 2017 4th NAFOSTED Conference on Information and Computer Science.