Decoding of Ternary Error Correcting Output Codes

Error correcting output codes (ECOC) represent a successful extension of binary classifiers to address the multiclass problem. Lately, the ECOC framework was extended from the binary to the ternary case to allow classes to be ignored by a certain classifier, allowing in this way to increase the number of possible dichotomies to be selected. Nevertheless, the effect of the zero symbol by which dichotomies exclude certain classes from consideration has not been previously enough considered in the definition of the decoding strategies. In this paper, we show that by a special treatment procedure of zeros, and adjusting the weights at the rest of coded positions, the accuracy of the system can be increased. Besides, we extend the main state-of-art decoding strategies from the binary to the ternary case, and we propose two novel approaches: Laplacian and Pessimistic Beta Density Probability approaches. Tests on UCI database repository (with different sparse matrices containing different percentages of zero symbol) show that the ternary decoding techniques proposed outperform the standard decoding strategies.

[1]  J. Friedman Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[2]  Christopher M. Bishop,et al.  Classification and regression , 1997 .

[3]  V. Vapnik Estimation of Dependences Based on Empirical Data , 2006 .

[4]  Reza Ghaderi,et al.  Coding and decoding strategies for multi-class learning problems , 2003, Inf. Fusion.

[5]  Thomas G. Dietterich,et al.  Error-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs , 1991, AAAI.

[6]  Sergio Escalera,et al.  ECOC-ONE: A Novel Coding and Decoding Strategy , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[7]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[8]  Koby Crammer,et al.  On the Learnability and Design of Output Codes for Multiclass Problems , 2002, Machine Learning.

[9]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[10]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[11]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[12]  Robert Tibshirani,et al.  Classification by Pairwise Coupling , 1997, NIPS.

[13]  Jordi Vitrià,et al.  Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Sergio Escalera,et al.  Forest Extension of Error Correcting Output Codes and Boosted Landmarks , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[15]  T Windeatt,et al.  CODING AND DECODING FOR MULTI-CLASS LEARNING PROBLEMS , 2003 .