Constructing the Shortest ECOC for Fast Multi-classification

Error-correcting output codes (ECOC) is an effective method to perform multi-classification via decomposing a multi-classification problem into many binary classification tasks, and then integrating the outputs of the subtasks into a whole decision. The researches on applying ECOC to multi-classification mainly focus on how to improve the correcting ability of output codes and how to enhance the classification effectiveness of ECOC. This paper addresses a simple but interesting and significant case of ECOC, the shortest ECOC, to perform fast multi-classification at the cost of sacrificing a very small classification precision. The strategy of balancing the positive and negative examples for each binary classifier of ECOC and the method of finding the optimal permutation of all original classes are further given. Preliminary experimental results show, the shortest ECOC uses fewest binary classifiers but can still obtain comparable or close classification precisions with several traditional encoding methods of ECOC.

[1]  Koby Crammer,et al.  On the Learnability and Design of Output Codes for Multiclass Problems , 2002, Machine Learning.

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  Reza Ghaderi,et al.  Coding and decoding strategies for multi-class learning problems , 2003, Inf. Fusion.

[4]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[5]  Nicolás García-Pedrajas,et al.  Evolving Output Codes for Multiclass Problems , 2008, IEEE Transactions on Evolutionary Computation.

[6]  Jordi Vitrià,et al.  Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Sergio Escalera,et al.  Subclass Problem-Dependent Design for Error-Correcting Output Codes , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[9]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[10]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[11]  Michel Gendreau,et al.  An Introduction to Tabu Search , 2003, Handbook of Metaheuristics.

[12]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[13]  Cheng-Chew Lim,et al.  Target detection in radar imagery using support vector machines with training size biasing , 2000 .

[14]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.