One-Class Classification: A Survey

One-Class Classification (OCC) is a special case of multi-class classification, where data observed during training is from a single positive class. The goal of OCC is to learn a representation and/or a classifier that enables recognition of positively labeled queries during inference. This topic has received considerable amount of interest in the computer vision, machine learning and biometrics communities in recent years. In this article, we provide a survey of classical statistical and recent deep learning-based OCC methods for visual recognition. We discuss the merits and drawbacks of existing OCC approaches and identify promising avenues for research in this field. In addition, we present a discussion of commonly used datasets and evaluation metrics for OCC.

[1]  Kian Hsiang Low,et al.  GEE: A Gradient-based Explainable Variational Autoencoder for Network Anomaly Detection , 2019, 2019 IEEE Conference on Communications and Network Security (CNS).

[2]  Vishal M. Patel,et al.  Active Authentication using an Autoencoder regularized CNN-based One-Class Classifier , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[3]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[4]  Anil K. Jain,et al.  Generalizing Fingerprint Spoof Detector: Learning a One-Class Classifier , 2019, 2019 International Conference on Biometrics (ICB).

[5]  Michael I. Jordan,et al.  Minimax Probability Machine , 2001, NIPS.

[6]  Hironobu Fujiyoshi,et al.  Attention Branch Network: Learning of Attention Mechanism for Visual Explanation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ran El-Yaniv,et al.  Deep Anomaly Detection Using Geometric Transformations , 2018, NeurIPS.

[8]  Daniel P. Robinson,et al.  Provable Self-Representation Based Outlier Detection in a Union of Subspaces , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Alan L. Yuille,et al.  Feature Denoising for Improving Adversarial Robustness , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[12]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[13]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Antonio J. Plaza,et al.  One-Class Classification of Remote Sensing Images Using Kernel Sparse Representation , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Raghavendra Chalapathy University of Sydney,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.

[16]  Shehroz S. Khan,et al.  A Survey of Recent Trends in One Class Classification , 2009, AICS.

[17]  Shehroz S. Khan,et al.  One-class classification: taxonomy of study and review of techniques , 2013, The Knowledge Engineering Review.

[18]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[19]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[20]  Vishal M. Patel,et al.  Anomaly Detection-Based Unknown Face Presentation Attack Detection , 2020, 2020 IEEE International Joint Conference on Biometrics (IJCB).

[21]  Vishal M. Patel,et al.  One-Class Convolutional Neural Network , 2019, IEEE Signal Processing Letters.

[22]  John W. Sammon,et al.  An Optimal Set of Discriminant Vectors , 1975, IEEE Transactions on Computers.

[23]  Sébastien Marcel,et al.  On Effectiveness of Anomaly Detection Approaches against Unseen Presentation Attacks in Face Anti-spoofing , 2018, 2018 International Conference on Biometrics (ICB).

[24]  Rama Chellappa,et al.  Design of Non-Linear Kernel Dictionaries for Object Recognition , 2013, IEEE Transactions on Image Processing.

[25]  Alexander Binder,et al.  Deep Semi-Supervised Anomaly Detection , 2019, ICLR.

[26]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[27]  Rahil Garnavi,et al.  Generative OpenMax for Multi-Class Open Set Classification , 2017, BMVC.

[28]  Vishal M. Patel,et al.  Learning Deep Features for One-Class Classification , 2018, IEEE Transactions on Image Processing.

[29]  Xiaoming Liu,et al.  Transfer learning with one-class data , 2014, Pattern Recognit. Lett..

[30]  Vishal M. Patel,et al.  C2AE: Class Conditioned Auto-Encoder for Open-Set Recognition , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Shifeng Chen,et al.  P-KDGAN: Progressive Knowledge Distillation with GANs for One-class Novelty Detection , 2020, IJCAI.

[32]  Weng-Keen Wong,et al.  Open Set Learning with Counterfactual Images , 2018, ECCV.

[33]  Vishal M. Patel,et al.  Dual-Minimax Probability Machines for One-class Mobile Active Authentication , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[34]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[36]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[37]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[38]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[39]  Marius Kloft,et al.  Explainable Deep One-Class Classification , 2020, ICLR.

[40]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[41]  Thorsten Joachims,et al.  Learning a Distance Metric from Relative Comparisons , 2003, NIPS.

[42]  Bernhard Schölkopf,et al.  Prior Knowledge in Support Vector Kernels , 1997, NIPS.

[43]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[44]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[46]  Vishal M. Patel,et al.  Sparse Representation-Based Open Set Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[48]  Seung-Ik Lee,et al.  Old Is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[50]  Bin Yang,et al.  One-Class Feature Learning Using Intra-Class Splitting , 2018, 2019 27th European Signal Processing Conference (EUSIPCO).

[51]  Bing Liu,et al.  HRN: A Holistic Approach to One Class Learning , 2020, NeurIPS.

[52]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[54]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[55]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[56]  Zuren Feng,et al.  Video Anomaly Detection Using Ensemble One-Class Classifiers , 2018, 2018 37th Chinese Control Conference (CCC).

[57]  Ali Farhadi,et al.  Object-Centric Anomaly Detection by Attribute-Based Reasoning , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Vineeth N. Balasubramanian,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[59]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[60]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[61]  Terrance E. Boult,et al.  Towards Open Set Deep Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[63]  Stanislav Pidhorskyi,et al.  Generative Probabilistic Novelty Detection with Adversarial Autoencoders , 2018, NeurIPS.

[64]  Gilad Lerman,et al.  Robust Subspace Recovery Layer for Unsupervised Anomaly Detection , 2019, ICLR.

[65]  Simone Calderara,et al.  Latent Space Autoregression for Novelty Detection , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[67]  Joost van de Weijer,et al.  Metric Learning for Novelty and Anomaly Detection , 2018, BMVC.

[68]  Ramesh Nallapati,et al.  OCGAN: One-Class Novelty Detection Using GANs With Constrained Latent Representations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Longbing Cao,et al.  Deep Learning for Anomaly Detection: A Review , 2020, ArXiv.

[70]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[71]  Terrance E. Boult,et al.  Probability Models for Open Set Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[73]  Carsten Steger,et al.  MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[75]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[76]  Toby P. Breckon,et al.  GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training , 2018, ACCV.

[77]  Joachim Denzler,et al.  Kernel Null Space Methods for Novelty Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[78]  Vishal M. Patel,et al.  Deep Transfer Learning for Multiple Class Novelty Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[79]  Sumit Srivastava,et al.  Domain Adaptation for Automatic OLED Panel Defect Detection Using Adaptive Support Vector Data Description , 2017, International Journal of Computer Vision.

[80]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[81]  Josef Kittler,et al.  Combining Multiple one-class Classifiers for Anomaly based Face Spoofing Attack Detection , 2019, 2019 International Conference on Biometrics (ICB).

[82]  Anoop Cherian,et al.  GODS: Generalized One-Class Discriminative Subspaces for Anomaly Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[83]  C. Steger,et al.  Uninformed Students: Student-Teacher Anomaly Detection With Discriminative Latent Embeddings , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[84]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[85]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[86]  Rama Chellappa,et al.  Visual Domain Adaptation: A survey of recent advances , 2015, IEEE Signal Processing Magazine.

[87]  Shivangi Yadav,et al.  Relativistic Discriminator: A One-Class Classifier for Generalized Iris Presentation Attack Detection , 2020, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[88]  Nikos Komodakis,et al.  Unsupervised Representation Learning by Predicting Image Rotations , 2018, ICLR.

[89]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[90]  Heiko Hoffmann,et al.  Kernel PCA for novelty detection , 2007, Pattern Recognit..

[91]  Ian H. Witten,et al.  One-Class Classification by Combining Density and Class Probability Estimation , 2008, ECML/PKDD.

[92]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[93]  Michael I. Jordan,et al.  Robust Novelty Detection with Single-Class MPM , 2002, NIPS.

[94]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[95]  Bir Bhanu,et al.  Towards Visually Explaining Variational Autoencoders , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[96]  Hamid R. Rabiee,et al.  ARAE: Adversarially Robust Training of Autoencoders Improves Novelty Detection , 2020, Neural Networks.

[97]  Mahmood Fathy,et al.  Adversarially Learned One-Class Classifier for Novelty Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[98]  Zachary Chase Lipton.  The mythos of model interpretability , 2016, ACM Queue.

[99]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[100]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.