One-class and Multi-class classier combining for ill-dened problems

Elect. Eng., Maths and Comp. Sc., Delft University of Technology, Delft, TheNetherlandsAbstractClassi er performance can be severely a ected when new unseen classes are present,or the conditional distribution of one of the classes changes. Both classi cationand rejection performance should be considered. The distance-based reject-optionis commonly used in this situation. The model chosen for classi cation is used inrejection. A model chosen emphasising classi cation performance may be at theexpense of rejection performance, and the opposite also holds. In this paper a clas-si cation strategy is presented, consisting of the sequential combining of one-classand multi-class classi ers. Two variants of this classi er are presented. These strate-gies have the exibility to select distinct models for classi cation and rejection, andoperate on local regions of the data to emphasise either classi cation or rejection.An evaluation methodology is presented, and a number of real-world experimentsare carried out that illustrate the potential of this approach, showing that in somesituations they can improve over the reject-option.Key words: Ill-de ned classes, Reject-option, Classi er-combining, One-classclassi cation, Multi-stage Corresponding author. Tel.: +31 (0)15 27 88433; fax: +31 (0)15 27 81843; P.O.Box 5031 2600 GA Delft, Mekelweg 4 2628CD Delft, The NetherlandsEmail addresses: t.c.w.landgrebe@ewi.tudelft.nl(Thomas C.W.Landgrebe), d.m.j.tax@ewi.tudelft.nl(David M.J. Tax),p.paclik@ewi.tudelft.nl(Pavel Paclk), r.p.w.duin@ewi.tudelft.nl(RobertP.W. Duin), colin.andrew@ieee.org(Colin M. Andrew).Preprint submitted to Elsevier Science 14 October 2004

[1]  Marcel Worring,et al.  Face detection by aggregated Bayesian network classifiers , 2001, Pattern Recognit. Lett..

[2]  Andrew Webb,et al.  Classifier Design for Population and Sensor Drift , 2004, SSPR/SPR.

[3]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[4]  Robert P. W. Duin,et al.  A combining strategy for ill-defined problems , 2006 .

[5]  Robert P. W. Duin,et al.  Building Road-Sign Classifiers Using a Trainable Similarity Measure , 2006, IEEE Transactions on Intelligent Transportation Systems.

[6]  Robert P. W. Duin,et al.  Cost-Based Classifier Evaluation for Imbalanced Problems , 2004, SSPR/SPR.

[7]  Bernard Dubuisson,et al.  A statistical decision rule with incomplete knowledge about classes , 1993, Pattern Recognit..

[8]  David M. J. Tax,et al.  One-class classification , 2001 .

[9]  Robert P. W. Duin,et al.  Uniform Object Generation for Optimizing One-class Classifiers , 2002, J. Mach. Learn. Res..

[10]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[11]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[12]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[13]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[14]  Hiroshi Sako,et al.  Performance evaluation of pattern classifiers for handwritten character recognition , 2002, International Journal on Document Analysis and Recognition.

[15]  Robert P. W. Duin,et al.  The combining classifier: to train or not to train? , 2002, Object recognition supported by user interaction for service robots.

[16]  Robert P. W. Duin,et al.  Robust machine fault detection with independent component analysis and support vector data description , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[17]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .