Error Correcting Output Codes Using Genetic Algorithm-Based Decoding

Error correcting output codes (ECOC) is one of the most valuable methods in building multiple classifier systems. This method decomposes a multiclass problem into a number of simpler binary sub-problems called dichotomies. The simplest methods of reconstruction of ECOC ensemble are Hamming and Margin decoding. Thay ignore the difference of dichotomies that lead to different base classifiers. In this paper, we give a new and general technique for combining classifiers that does not suffer from this defect. We use weights for adjusting the distance of base classifier outputs from the labels of existing classes. Optimal weights are determined by a proposed Genetic algorithm-based method which is the popular one of evolutionary Algorithms. Experimental results on two benchmark datasets and two different algorithms as the base classifiers show the robustness of the proposed decoding method with respect to the previously introduced decoding methods.

[1]  CodingEun Bae Kong.  Probability Estimation via Error-Correcting Output , 1997 .

[2]  F. Lemmermeyer.  Error-correcting Codes , 2005 .

[3]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[4]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[5]  Kamal A. Ali,et al.  On the Link between Error Correlation and Error Reduction in Decision Tree Ensembles , 1995 .

[6]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[7]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[8]  W. W. Peterson,et al.  Error-Correcting Codes. , 1962 .

[9]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[10]  Jiri Matas,et al.  Face verification via error correcting output codes , 2003, Image Vis. Comput..

[11]  Cesare Alippi,et al.  Genetic-algorithm programming environments , 1994, Computer.

[12]  Reza Ghaderi,et al.  Least Squares and Estimation Measures via Error Correcting Output Code , 2001, Multiple Classifier Systems.

[13]  Thomas G. Dietterich,et al.  Error-Correcting Output Coding Corrects Bias and Variance , 1995, ICML.

[14]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[15]  Nima Hatami.  Thinned ECOC Decomposition for Gene Expression Based Cancer Classification , 2008, 2008 Eighth International Conference on Intelligent Systems Design and Applications.

[16]  Reza Ebrahimpour,et al.  Combining Multiple Classifiers: Diversify with Boosting and Combining by Stacking , 2007 .

[17]  Reza Ghaderi,et al.  Binary labelling and decision-level fusion , 2001, Inf. Fusion.

[18]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[19]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[20]  Reza Ebrahimpour,et al.  ECOC-based training of neural networks for face recognition , 2008, 2008 IEEE Conference on Cybernetics and Intelligent Systems.

[21]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[22]  E. Kim,et al.  On ECOC as Binary Ensemble Classifiers , 2005, MLDM.

[23]  Bogdan Gabrys,et al.  Genetic algorithms in classifier fusion , 2006, Appl. Soft Comput..

[24]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[25]  Reza Ghaderi,et al.  Coding and decoding strategies for multi-class learning problems , 2003, Inf. Fusion.

[26]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..