On Diffusion Modeling for Anomaly Detection

Known for their impressive performance in generative modeling, diffusion models are attractive candidates for density-based anomaly detection. This paper investigates different variations of diffusion modeling for unsupervised and semi-supervised anomaly detection. In particular, we find that Denoising Diffusion Probability Models (DDPM) are performant on anomaly detection benchmarks yet computationally expensive. By simplifying DDPM in application to anomaly detection, we are naturally led to an alternative approach called Diffusion Time Estimation (DTE). DTE estimates the distribution over diffusion time for a given input and uses the mode or mean of this distribution as the anomaly score. We derive an analytical form for this density and leverage a deep neural network to improve inference efficiency. Through empirical evaluations on the ADBench benchmark, we demonstrate that all diffusion-based anomaly detection methods perform competitively for both semi-supervised and unsupervised settings. Notably, DTE achieves orders of magnitude faster inference time than DDPM, while outperforming it on this benchmark. These results establish diffusion-based anomaly detection as a scalable alternative to traditional methods and recent deep-learning techniques for standard unsupervised and semi-supervised anomaly detection settings.

[1]  Zuxuan Wu,et al.  DiffusionAD: Denoising Diffusion for Anomaly Detection , 2023, ArXiv.

[2]  Artem Babenko,et al.  TabDDPM: Modelling Tabular Data with Diffusion Models , 2022, ICML.

[3]  Yue Zhao,et al.  ADBench: Anomaly Detection Benchmark , 2022, NeurIPS.

[4]  Chris G. Willcocks,et al.  AnoDDPM: Anomaly Detection with Denoising Diffusion Probabilistic Models using Simplex Noise , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Prafulla Dhariwal,et al.  Hierarchical Text-Conditional Image Generation with CLIP Latents , 2022, ArXiv.

[6]  P. Cattin,et al.  Diffusion Models for Medical Anomaly Detection , 2022, MICCAI.

[7]  George H. Chen,et al.  ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions , 2022, IEEE Transactions on Knowledge and Data Engineering.

[8]  Wee Siong Ng,et al.  LUNAR: Unifying Local Outlier Detection Methods via Graph Neural Networks , 2021, AAAI.

[9]  Katherine Fraser,et al.  Challenges for unsupervised anomaly detection in particle physics , 2021, Journal of High Energy Physics.

[10]  Artem Babenko,et al.  Revisiting Deep Learning Models for Tabular Data , 2021, NeurIPS.

[11]  Abhishek Kumar,et al.  Score-Based Generative Modeling through Stochastic Differential Equations , 2020, ICLR.

[12]  Thomas G. Dietterich,et al.  A Unifying Review of Deep and Shallow Anomaly Detection , 2020, Proceedings of the IEEE.

[13]  Cezar Ionescu,et al.  COPOD: Copula-Based Outlier Detection , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[14]  Pieter Abbeel,et al.  Denoising Diffusion Probabilistic Models , 2020, NeurIPS.

[15]  Yedid Hoshen,et al.  Classification-Based Anomaly Detection for General Data , 2020, ICLR.

[16]  Yongsub Lim,et al.  RaPP: Novelty Detection with Reconstruction along Projection Pathway , 2020, ICLR.

[17]  Harsha Vardhan Simhadri,et al.  DROCC: Deep Robust One-Class Classification , 2020, ICML.

[18]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[19]  Chalapathy Raghavendra,et al.  Deep Learning for Anomaly Detection , 2019 .

[20]  Meng Wang,et al.  Generative Adversarial Active Learning for Unsupervised Outlier Detection , 2018, IEEE Transactions on Knowledge and Data Engineering.

[21]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[22]  Randy C. Paffenroth,et al.  Anomaly Detection with Robust Deep Autoencoders , 2017, KDD.

[23]  Christopher Leckie,et al.  High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning , 2016, Pattern Recognit..

[24]  A. Mahmood,et al.  A survey of anomaly detection techniques in financial domain , 2016, Future Gener. Comput. Syst..

[25]  Tomás Pevný,et al.  Loda: Lightweight on-line detector of anomalies , 2016, Machine Learning.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Gang Hua,et al.  Learning Discriminative Reconstructions for Unsupervised Outlier Removal , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[29]  Surya Ganguli,et al.  Deep Unsupervised Learning using Nonequilibrium Thermodynamics , 2015, ICML.

[30]  Takehisa Yairi,et al.  Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction , 2014, MLSDA'14.

[31]  Borko Furht,et al.  Sensor fault and patient anomaly detection and classification in medical wireless sensor networks , 2013, 2013 IEEE International Conference on Communications (ICC).

[32]  Gentiane Haesbroeck,et al.  Outliers detection with the minimum covariance determinant estimator in practice , 2009 .

[33]  Vipin Kumar,et al.  Anomaly detection: A survey , 2009, CSUR.

[34]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[35]  Y. Kawahara,et al.  Telemetry-mining: a machine learning approach to anomaly detection and fault diagnosis for space systems , 2006, 2nd IEEE International Conference on Space Mission Challenges for Information Technology (SMC-IT'06).

[36]  Vipin Kumar,et al.  Feature bagging for outlier detection , 2005, KDD '05.

[37]  Zengyou He,et al.  Discovering cluster-based local outliers , 2003, Pattern Recognit. Lett..

[38]  Christos Faloutsos,et al.  LOCI: fast outlier detection using the local correlation integral , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[39]  Jian Tang,et al.  Enhancing Effectiveness of Outlier Detections for Low Density Patterns , 2002, PAKDD.

[40]  Sridhar Ramaswamy,et al.  Efficient algorithms for mining outliers from large data sets , 2000, SIGMOD '00.

[41]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[42]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[43]  Lior Wolf,et al.  Anomaly Detection for Tabular Data with Internal Contrastive Learning , 2022, ICLR.

[44]  Matteo Terzi,et al.  Anomaly Detection Approaches for Semiconductor Manufacturing , 2017 .

[45]  Mohiuddin Ahmed,et al.  A survey of network anomaly detection techniques , 2016, J. Netw. Comput. Appl..

[46]  Sungzoon Cho,et al.  Variational Autoencoder based Anomaly Detection using Reconstruction Probability , 2015 .

[47]  Sandeep Sharma,et al.  Anomaly Detection in Medical Wireless Sensor Networks using Machine Learning Algorithms , 2015 .

[48]  Andreas Dengel,et al.  Histogram-based Outlier Score (HBOS): A fast Unsupervised Anomaly Detection Algorithm , 2012 .

[49]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[50]  M. Shyu,et al.  A Novel Anomaly Detection Scheme Based on Principal Component Classifier , 2003 .

[51]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .