An improved error-correcting output coding framework with kernel-based decoding

This paper presents a novel framework of error-correcting output coding (ECOC) addressing the problem of multi-class classification. By weighting the output space of each base classifier which is trained independently, the distance function of decoding is adapted so that the samples are more discriminative. A criterion generated over the Extended Pair Samples (EPS) is proposed to train the weights of output space. We first conduct empirical studies on UCI datasets to verify the presented framework with four frequently used coding matrixes and then apply it in RoboCup domain to enhance the performance of agent control. Color-based image segmentation is also tested in this paper. Experimental results show that our supervised learned decoding scheme improves the accuracy of classification significantly and betters the ball control of agents in a soccer game after learning from experience. Some properties still hold in the new framework: any classifier, as well as distance function, is still applicable. Besides, it is allowed in the new framework to implement nonlinear decoding processes using kernel tricks.

[1]  K. Kuhlmann.  Progress in Learning 3 vs . 2 , 2003 .

[2]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[3]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[4]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[5]  D W Aha,et al.  CLOUD CLASSIFICATION USING ERRORCORRECTING OUTPUT CODES , 1997 .

[6]  Adam L. Berger,et al.  ERROR-CORRECTING OUTPUT CODING FOR TEXT CLASSIFICATION , 1999 .

[7]  Paolo Frasconi,et al.  From Margins to Probabilities in Multiclass Learning Problems , 2002, ECAI.

[8]  Paolo Frasconi,et al.  New results on error correcting output codes of kernel machines , 2004, IEEE Transactions on Neural Networks.

[9]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[10]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[11]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[12]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[13]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[14]  Robert E. Schapire,et al.  Using output codes to boost multiclass learning problems , 1997, ICML.

[15]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[16]  Frank van Harmelen,et al.  Proceedings of the 15th European Conference on Artificial Intelligence , 2002 .

[17]  Thomas G. Dietterich,et al.  Machine Learning Bias, Statistical Bias, and Statistical Variance of Decision Tree Algorithms , 2008 .

[18]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[19]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[20]  Koby Crammer,et al.  On the Learnability and Design of Output Codes for Multiclass Problems , 2002, Machine Learning.

[21]  Peter Stone,et al.  Progress in Learning 3 vs. 2 Keepaway , 2003, RoboCup.

[22]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..