Selection of Accurate and Robust Classification Model for Binary Classification Problems

In this paper we aim to investigate the trade off in selection of an accurate, robust and cost-effective classification model for binary classification problem. With empirical observation we present the evaluation of one-class and two-class classification model. We have experimented with four two-class and one-class classifier models on five UCI datasets. We have evaluated the classification models with Receiver Operating Curve (ROC), Cross validation Error and pair-wise measure Q statistics. Our finding is that in the presence of large amount of relevant training data the two-class classifiers perform better than one-class classifiers for binary classification problem. It is due to the ability of the two class classifier to use negative data samples in its decision. In scenarios when sufficient training data is not available the one-class classification model performs better.

[1]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[2]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[3]  Sungzoon Cho,et al.  Novelty Detection Approach for Keystroke Dynamics Identity Verification , 2003, IDEAL.

[4]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[5]  李幼升,et al.  Ph , 1989 .

[6]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[7]  Moshe Koppel,et al.  Authorship verification as a one-class classification problem , 2004, ICML.

[8]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[9]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[10]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[11]  Robert P. W. Duin,et al.  Support vector domain description , 1999, Pattern Recognit. Lett..

[12]  Ran El-Yaniv,et al.  Towards Behaviometric Security Systems: Learning to Identify a Typist , 2003, PKDD.

[13]  Fabio Roli,et al.  Multiple Classifier Systems, 9th International Workshop, MCS 2010, Cairo, Egypt, April 7-9, 2010. Proceedings , 2010, MCS.

[14]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[15]  David G. Stork,et al.  Pattern Classification , 1973 .

[16]  Robert P. W. Duin,et al.  Combining One-Class Classifiers , 2001, Multiple Classifier Systems.

[17]  Hendrik Blockeel,et al.  Knowledge Discovery in Databases: PKDD 2003 , 2003, Lecture Notes in Computer Science.

[18]  Chengjun Liu,et al.  Robust coding schemes for indexing and retrieval from large face databases , 2000, IEEE Trans. Image Process..

[19]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[20]  David M. J. Tax,et al.  One-class classification , 2001 .