Growing a multi-class classifier with a reject option

In many classification problems objects should be rejected when the confidence in their classification is too low. An example is a face recognition problem where the faces of a selected group of people have to be classified, but where all other faces and non-faces should be rejected. These problems are typically solved by estimating the class densities and assigning an object to the class with the highest posterior probability. The total probability density is thresholded to detect the outliers. Unfortunately, this procedure does not easily allow for class-dependent thresholds, or for class models that are not based on probability densities but on distances. In this paper we propose a new heuristic to combine any type of one-class models for solving the multi-class classification problem with outlier rejection. It normalizes the average model output per class, instead of the more common non-linear transformation of the distances. It creates the possibility to adjust the rejection threshold per class, and also to combine class models that are not (all) based on probability densities and to add class models without affecting the boundaries of existing models. Experiments show that for several classification problems using class-specific models significantly improves the performance.

[1]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[2]  David M. J. Tax,et al.  One-class classification , 2001 .

[3]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[4]  Marco Riani,et al.  The Ordering of Spatial Data and the Detection of Multiple Outliers , 1999 .

[5]  Sridhar Ramaswamy,et al.  Efficient algorithms for mining outliers from large data sets , 2000, SIGMOD '00.

[6]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[7]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[8]  David G. Stork,et al.  Pattern Classification , 1973 .

[9]  Robert P. W. Duin,et al.  Data domain description using support vectors , 1999, ESANN.

[10]  P. Malvache,et al.  Computer aided system diagnostic with an incomplete learning set , 1985 .

[11]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[12]  Jin Young Choi,et al.  SVDD-based method for Face Recognition System , 2006 .

[13]  Robert P. W. Duin,et al.  On the Choice of Smoothing Parameters for Parzen Estimators of Probability Density Functions , 1976, IEEE Transactions on Computers.

[14]  Kai-Tai Fang,et al.  Multiple outlier detection in multivariate data using projection pursuit techniques , 2000 .

[15]  Robert P. W. Duin,et al.  Uniform Object Generation for Optimizing One-class Classifiers , 2002, J. Mach. Learn. Res..

[16]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[17]  Stephen Marsland,et al.  On-Line Novelty Detection through self-organisation with application to inspection robotics , 2001 .

[18]  Bernard Dubuisson,et al.  A statistical decision rule with incomplete knowledge about classes , 1993, Pattern Recognit..

[19]  Laurie Davies,et al.  The identification of multiple outliers , 1993 .

[20]  李幼升,et al.  Ph , 1989 .

[21]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[22]  Michael Brady,et al.  Novelty detection for the identification of masses in mammograms , 1995 .

[23]  L. Baker,et al.  A Hierarchical Probabilistic Model for Novelty Detection in Text , 1999, NIPS 1999.

[24]  Fabio Roli,et al.  Reject option with multiple thresholds , 2000, Pattern Recognit..

[25]  Nathalie Japkowicz,et al.  A Novelty Detection Approach to Classification , 1995, IJCAI.