Latent Outlier Exposure for Anomaly Detection with Contaminated Data

Anomaly detection aims at identifying data points that show systematic deviations from the major-ity of data in an unlabeled dataset. A common assumption is that clean training data (free of anomalies) is available, which is often violated in practice. We propose a strategy for training an anomaly detector in the presence of unlabeled anomalies that is compatible with a broad class of models. The idea is to jointly infer binary labels to each datum (normal vs. anomalous) while updating the model parameters. Inspired by outlier exposure (Hendrycks et al., 2018) that con-siders synthetically created, labeled anomalies, we thereby use a combination of two losses that share parameters: one for the normal and one for the anomalous data. We then iteratively proceed with block coordinate updates on the parameters and the most likely (latent) labels. Our experiments with several backbone models on three image datasets, 30 tabular data sets, and a video anomaly detection benchmark showed consistent and significant improvements over the baselines.

[1]  Maja R. Rudolph,et al.  Raising the Bar in Graph-level Anomaly Detection , 2022, IJCAI.

[2]  F. Kabanza,et al.  A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection Algorithms , 2022, ArXiv.

[3]  Maja R. Rudolph,et al.  Detecting Anomalies within Time Series using Local Neural Transformations , 2022, ArXiv.

[4]  Jocelyn Chanussot,et al.  Unsupervised Outlier Detection using Memory and Contrastive Learning , 2021, ArXiv.

[5]  Lior Wolf,et al.  Anomaly Detection for Tabular Data with Internal Contrastive Learning , 2022, ICLR.

[6]  Tomas Pfister,et al.  CutPaste: Self-Supervised Learning for Anomaly Detection and Localization , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Fa-Ting Hong,et al.  MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Maja R. Rudolph,et al.  Neural Transformation Learning for Deep Anomaly Detection Beyond Images , 2021, ICML.

[9]  Chun-Liang Li,et al.  Learning and Evaluating Representations for Deep One-class Classification , 2020, ICLR.

[10]  Yedid Hoshen,et al.  PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Thomas G. Dietterich,et al.  A Unifying Review of Deep and Shallow Anomaly Detection , 2020, Proceedings of the IEEE.

[12]  Chen-Yu Lee,et al.  Self-Trained One-class Classification for Unsupervised Anomaly Detection , 2021, ArXiv.

[13]  Romaric Audigier,et al.  PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization , 2020, ICPR Workshops.

[14]  Yedid Hoshen,et al.  Classification-Based Anomaly Detection for General Data , 2020, ICLR.

[15]  Chunhua Shen,et al.  Self-Trained Deep Ordinal Regression for End-to-End Video Anomaly Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Dawn Song,et al.  Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty , 2019, NeurIPS.

[17]  Carsten Steger,et al.  MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Bernd Bischl,et al.  Robust Anomaly Detection in Images using Adversarial Autoencoders , 2019, ECML/PKDD.

[19]  Thomas G. Dietterich,et al.  Deep Anomaly Detection with Outlier Exposure , 2018, ICLR.

[20]  Yee Whye Teh,et al.  Do Deep Generative Models Know What They Don't Know? , 2018, ICLR.

[21]  Jianping Yin,et al.  Effective End-to-end Unsupervised Outlier Detection via Inlier Priority of Discriminative Network , 2019, NeurIPS.

[22]  Marius Kloft,et al.  Image Anomaly Detection with Generative Adversarial Networks , 2018, ECML/PKDD.

[23]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[24]  Ran El-Yaniv,et al.  Deep Anomaly Detection Using Geometric Transformations , 2018, NeurIPS.

[25]  Ender Konukoglu,et al.  Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders , 2018, ArXiv.

[26]  Yusha Liu,et al.  Classifier Two Sample Test for Video Anomaly Detections , 2018, BMVC.

[27]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[28]  Radu Tudor Ionescu,et al.  Unmasking the Abnormal Events in Video , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Francesco Piazza,et al.  Acoustic novelty detection with adversarial autoencoders , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[30]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[31]  Martial Hebert,et al.  A Discriminative Framework for Anomaly Detection in Large Videos , 2016, ECCV.

[32]  Gang Hua,et al.  Learning Discriminative Reconstructions for Unsupervised Outlier Removal , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[34]  Alexander Binder,et al.  Learning and Evaluation in Presence of Non-i.i.d. Label Noise , 2014, AISTATS.

[35]  Karsten M. Borgwardt,et al.  Rapid Distance-Based Outlier Detection via Sampling , 2013, NIPS.

[36]  Fei Tony Liu,et al.  Isolation-Based Anomaly Detection , 2012, TKDD.

[37]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[38]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[39]  Bing Liu,et al.  Learning with Positive and Unlabeled Examples Using Weighted Logistic Regression , 2003, ICML.

[40]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .