One class proximal support vector machines

Recently in Dufrenois 1, a new Fisher type contrast measure has been proposed to extract a target population in a dataset contaminated by outliers. Although mathematically sound, this work presents some further shortcomings in both the formalism and the field of use. First, we propose to re-express this problem from the formalism of proximal support vector machines as introduced in Mangasarian and Wild 2. This change is far from harmless since it introduces a suited writing for solving the problem. Another limiting factor of the method is that its performance relies on the assumption that the density between the target and outliers are different. This consideration can easily prove to be over-optimistic for real world datasets making the method unreliable, at least directly. The computation of the decision boundary is a time consuming part of the algorithm since it is based on solving a generalized eigenvalue problem (GEP). This method is therefore limited to medium sized data sets. In this paper, we propose appropriate strategies to unlock all these shortcomings and fully benefit from the interest of the approach. Firstly, we show under some conditions that generating appropriate artificial outliers allows to stay within the constraints of the method and thus enlarges the conditions of use. Secondly, we show that the GEP can be advantageously replaced by a conjugate gradient solution (CG) significantly decreasing the computational cost. Lastly, the proposed algorithm is compared with recent novelty detectors on synthetic and real datasets. HighlightsWe study the extraction of a target population from a dataset contaminated by outliers.To this end, we propose a new Fisher type contrast measure.We reconsider this problem from the formalism of proximal support vector machines.An approximation of the contrast measure is done using a conjugate gradient method.No matrix inversion is needed which lowers the computational complexity.

[1]  Takeo Kanade,et al.  Discriminative cluster analysis , 2006, ICML.

[2]  Hans-Peter Kriegel,et al.  LoOP: local outlier probabilities , 2009, CIKM.

[3]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[4]  Takafumi Kanamori,et al.  Statistical outlier detection using direct density ratio estimation , 2011, Knowledge and Information Systems.

[5]  Robert P. W. Duin,et al.  One-Class LP Classifiers for Dissimilarity Representations , 2002, NIPS.

[6]  Joachim Denzler,et al.  Kernel Null Space Methods for Novelty Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  S. Chatterjee.  Sensitivity analysis in linear regression , 1988 .

[8]  David M. J. Tax,et al.  One-class classification , 2001 .

[9]  Franck Dufrenois,et al.  A One-Class Kernel Fisher Criterion for Outlier Detection , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[11]  Robert P. W. Duin,et al.  Uniform Object Generation for Optimizing One-class Classifiers , 2002, J. Mach. Learn. Res..

[12]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[13]  M. Kawanabe,et al.  Direct importance estimation for covariate shift adaptation , 2008 .

[14]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Song Wang,et al.  Tracking of object with SVM regression , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[16]  András Kocsor,et al.  Counter-Example Generation-Based One-Class Classification , 2007, ECML.

[17]  A. Fielding.  Sensitivity Analysis in Linear Regression , 1990 .

[18]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[19]  R. A. Mollineda,et al.  The class imbalance problem in pattern classification and learning , 2009 .

[20]  Ivor W. Tsang,et al.  Learning to Locate Relative Outliers , 2011, ACML.

[21]  Salvatore J. Stolfo,et al.  Using artificial anomalies to detect unknown and known network intrusions , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[22]  Jieping Ye,et al.  Discriminative K-means for Clustering , 2007, NIPS.

[23]  Heiko Hoffmann,et al.  Kernel PCA for novelty detection , 2007, Pattern Recognit..

[24]  Hans-Peter Kriegel,et al.  Generalized Outlier Detection with Flexible Kernel Density Estimates , 2014, SDM.

[25]  V. Roth.  Kernel Fisher Discriminants for Outlier Detection , 2006 .

[26]  A. Izenman,et al.  Outlier detection using the smallest kernel principal components , 2007 .

[27]  Jean-Charles Noyer,et al.  Formulating Robust Linear Regression Estimation as a One-Class LDA Criterion: Discriminative Hat Matrix , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[29]  Sheng-yi Jiang,et al.  Clustering-Based Outlier Detection Method , 2008, 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discovery.