Transformer-Based Anomaly Segmentation

The recent improvement in anomaly detection methods has prompted research into anomaly segmentation i.e. finding the pixels of the image that contain anomalies. In this paper, we investigate novel methods for unleashing the full power of pretrained features for anomaly segmentation. We first present a simple baseline that uses a pyramid of deep convolutional features and show that it significantly improves over the state-of-the-art methods, which are much more complex. One issue with the baseline approach is that it is unable to use the global context of the image effectively. We show that global attention-based methods are better able to utilize the global context. Specifically, we present an approach based on a multi-scale transformer architecture and show that it further improves performance. By analysing the attention maps, we find that they often detect anomalous image regions in a zero-shot fashion, providing some insight into the result. A qualitative evaluation of our method shows significant gains.

[1]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[3]  I. Jolliffe.  Principal Component Analysis , 2005 .

[4]  Rob Fergus,et al.  Differentiable Pooling for Hierarchical Feature Learning , 2012, ArXiv.

[5]  Takehisa Yairi,et al.  Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction , 2014, MLSDA'14.

[6]  Paolo Napoletano,et al.  Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity , 2018, Sensors.

[7]  Ran El-Yaniv,et al.  Deep Anomaly Detection Using Geometric Transformations , 2018, NeurIPS.

[8]  Carsten Steger,et al.  MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yedid Hoshen,et al.  Deep Nearest Neighbor Anomaly Detection , 2020, ArXiv.

[10]  Thomas G. Dietterich,et al.  Deep Anomaly Detection with Outlier Exposure , 2018, ICLR.

[11]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[12]  Lucas Beyer,et al.  Big Transfer (BiT): General Visual Representation Learning , 2020, ECCV.

[13]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[14]  Abhijit Mahalanobis,et al.  Attention Guided Anomaly Detection and Localization in Images , 2019, ArXiv.

[15]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[16]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[17]  Aleksandar Lazarevic,et al.  Outlier Detection with Kernel Density Functions , 2007, MLDM.

[18]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[19]  Georg Heigold,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ArXiv.

[20]  Paul Bergmann,et al.  Uninformed Students: Student-Teacher Anomaly Detection With Discriminative Latent Embeddings , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[22]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[23]  Yedid Hoshen,et al.  Classification-Based Anomaly Detection for General Data , 2020, ICLR.

[24]  Han Zhang,et al.  Self-Attention Generative Adversarial Networks , 2018, ICML.

[25]  Mark Chen,et al.  Generative Pretraining From Pixels , 2020, ICML.

[26]  Jeff Johnson,et al.  Billion-Scale Similarity Search with GPUs , 2017, IEEE Transactions on Big Data.

[27]  Marius Kloft,et al.  Image Anomaly Detection with Generative Adversarial Networks , 2018, ECML/PKDD.

[28]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Salvatore J. Stolfo,et al.  A Geometric Framework for Unsupervised Anomaly Detection , 2002, Applications of Data Mining in Computer Security.