A framework for imprecise robust one-class classification models

A framework for constructing robust one-class classification models is proposed in the paper. It is based on Walley’s imprecise extensions of contaminated models which produce a set of probability distributions of data points instead of a single empirical distribution. The minimax and minimin strategies are used to choose an optimal probability distribution from the set and to construct optimal separating functions. It is shown that an algorithm for computing optimal parameters is determined by extreme points of the probability set and is reduced to a finite number of standard SVM tasks with weighted data points. Important special cases of the models, including pari-mutuel, constant odd-ratio, contaminated models and Kolmogorov–Smirnov bounds are studied. Experimental results with synthetic and real data illustrate the proposed models.

[1]  Abdallah Bashir Musa.  Comparative study on classification performance between support vector machine and logistic regression , 2012, International Journal of Machine Learning and Cybernetics.

[2]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[3]  Tom Fawcett,et al.  Robust Classification for Imprecise Environments , 2000, Machine Learning.

[4]  Michael I. Jordan,et al.  Robust Novelty Detection with Single-Class MPM , 2002, NIPS.

[5]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[6]  Ujjwal Maulik,et al.  A novel semisupervised SVM for pixel classification of remote sensing imagery , 2012, Int. J. Mach. Learn. Cybern..

[7]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[8]  Yue Wang,et al.  Weighted support vector machine for data classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[9]  Theodore B. Trafalis,et al.  Robust support vector machines for classification and computational issues , 2007, Optim. Methods Softw..

[10]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[11]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[12]  P. P. Crump,et al.  Statistics and Experimental Design in Engineering and the Physical Sciences, Vol. I and II , 1978 .

[13]  Mário A. T. Figueiredo,et al.  Soft clustering using weighted one-class support vector machines , 2009, Pattern Recognit..

[14]  Zheng Gao,et al.  Multiple faults diagnosis in motion system based on SVM , 2012, Int. J. Mach. Learn. Cybern..

[15]  L. Wasserman.  All of Nonparametric Statistics , 2005 .

[16]  Christian P. Robert,et al.  The Bayesian choice , 1994 .

[17]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[18]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[19]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[20]  Lev V. Utkin and Frank P.A. Coolen.  On Reliability Growth Models Using Kolmogorov-Smirnov Bounds , 2011 .

[21]  C. L. Philip Chen,et al.  Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery , 2011, Int. J. Mach. Learn. Cybern..

[22]  Shie Mannor,et al.  Robustness and Regularization of Support Vector Machines , 2008, J. Mach. Learn. Res..

[23]  P. Walley.  Statistical Reasoning with Imprecise Probabilities , 1990 .

[24]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[25]  Colin Campbell,et al.  A Linear Programming Approach to Novelty Detection , 2000, NIPS.

[26]  A. Bartkowiak.  Anomaly, Novelty, One-Class Classification: A Comprehensive Introduction , 2011 .

[27]  Charles Bouveyron,et al.  Robust supervised classification with mixture models: Learning from data with uncertain labels , 2009, Pattern Recognit..

[28]  Jacek M. Zurada,et al.  A Class of Single-Class Minimax Probability Machines for Novelty Detection , 2007, IEEE Transactions on Neural Networks.

[29]  Koby Crammer,et al.  Robust Support Vector Machine Training via Convex Outlier Ablation , 2006, AAAI.

[30]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[31]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[32]  Don R. Hush,et al.  A Classification Framework for Anomaly Detection , 2005, J. Mach. Learn. Res..

[33]  Matthias C. M. Troffaes.  Decision making under uncertainty using imprecise probabilities , 2007, Int. J. Approx. Reason..

[34]  Michael I. Jordan,et al.  A Robust Minimax Approach to Classification , 2003, J. Mach. Learn. Res..

[35]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[36]  Shehroz S. Khan,et al.  A Survey of Recent Trends in One Class Classification , 2009, AICS.

[37]  Colin Campbell,et al.  Kernel methods: a survey of current techniques , 2002, Neurocomputing.

[38]  Laurent El Ghaoui,et al.  Robust Optimization , 2021, ICORES.

[39]  Massimiliano Pontil,et al.  Regularization and statistical learning theory for data analysis , 2002 .

[40]  Norman L. Johnson,et al.  Statistics and experimental design: in engineering and the physical science , 1965 .

[41]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[42]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[43]  Jinbo Bi,et al.  Support Vector Classification with Input Data Uncertainty , 2004, NIPS.

[44]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[45]  Jie Wang,et al.  Gaussian kernel optimization for pattern classification , 2009, Pattern Recognit..

[46]  Anthony C. Atkinson,et al.  Robust classification with categorical variables , 2006 .

[47]  L. Ghaoui,et al.  Robust Classification with Interval Data , 2003 .

[48]  I. Gilboa,et al.  Maxmin Expected Utility with Non-Unique Prior , 1989 .