A null space based one class kernel Fisher discriminant

Recently in [1], [2], a new kernel Fisher type contrast measure has been proposed to extract a target population in a data set contaminated by outliers. Although mathematically sound, this work presents some further shortcomings. First, the performance of the method relies on the assumption that the density between the target data and outliers is different. However, this consideration can easily prove to be over-optimistic for real world data sets making the method unreliable, at least directly. Secondly, this contrast measure is similar to a Generalized Rayleigh Quotient which is renowned for being sensitive to the small sample size problem. In this paper, we propose a null-space based version of their algorithm in order to unlock all these shortcomings and fully benefit from the interest of the approach. By this way, we show that the method can be used in a semi-supervised mode by considering that target objects can be beforehand collected and will serve as referentials to classify unseen objects. Experimental results on both synthetic and real data sets confirm the effectiveness of the proposed algorithm.

[1]  Joachim Denzler,et al.  Kernel Null Space Methods for Novelty Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[3]  G. Griffin,et al.  Caltech-256 Object Category Dataset , 2007 .

[4]  Mao Dun,et al.  Object Tracking Using Minimum-Spanning-Tree One-Class Classifier , 2016, 2016 9th International Symposium on Computational Intelligence and Design (ISCID).

[5]  Delin Chu,et al.  A new and fast implementation for null space based linear discriminant analysis , 2010, Pattern Recognit..

[6]  Joachim Denzler,et al.  One-class classification with Gaussian processes , 2010, Pattern Recognit..

[7]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[8]  Robert P. W. Duin,et al.  Minimum spanning tree based one-class classifier , 2009, Neurocomputing.

[9]  Jean-Charles Noyer,et al.  Formulating Robust Linear Regression Estimation as a One-Class LDA Criterion: Discriminative Hat Matrix , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[10]  David M. J. Tax,et al.  One-class classification , 2001 .

[11]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[12]  Franck Dufrenois,et al.  A One-Class Kernel Fisher Criterion for Outlier Detection , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Xiangyang Xue,et al.  Null Foley-Sammon transform , 2006, Pattern Recognit..

[14]  Takafumi Kanamori,et al.  Statistical outlier detection using direct density ratio estimation , 2011, Knowledge and Information Systems.

[15]  Ammar Belatreche,et al.  An experimental evaluation of novelty detection methods , 2014, Neurocomputing.

[16]  András Kocsor,et al.  Counter-Example Generation-Based One-Class Classification , 2007, ECML.

[17]  Jean-Charles Noyer,et al.  Generalized eigenvalue proximal support vector machines for outlier description , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[18]  Salvatore J. Stolfo,et al.  Using artificial anomalies to detect unknown and known network intrusions , 2003, Knowledge and Information Systems.

[19]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[20]  Robert P. W. Duin,et al.  Uniform Object Generation for Optimizing One-class Classifiers , 2002, J. Mach. Learn. Res..

[21]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[22]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[23]  Christos Faloutsos,et al.  OBE: Outlier by Example , 2004, PAKDD.

[24]  Jieping Ye,et al.  Null space versus orthogonal linear discriminant analysis , 2006, ICML '06.

[25]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[26]  Caroline Petitjean,et al.  One class random forests , 2013, Pattern Recognit..

[27]  F. Dufrenois,et al.  One class proximal support vector machines , 2016, Pattern Recognit..

[28]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.