Genetic algorithm-based neural error correcting output classifier

The present study elaborates a probabilistic framework of ECOC technique, via replacement of predesigned ECOC matrix by sufficiently large random codes. Further mathematical grounds of deploying random codes through probability formulations are part of novelty of this study. Random variants of ECOC techniques were applied in previous literatures, however, often failing to deliver sufficient theoretical proof of efficiency of random coding matrix. In this paper a Genetic Algorithm-based neural encoder with redefined operators is designed and trained. A variant of heuristic trimming of ECOC codewords is also deployed to acquire more satisfactory results. The efficacy of proposed approach was validated over a wide set of datasets of UCI Machine Learning Repository and compared against two conventional methods.

[1]  Yoshua Bengio,et al.  Exploring Strategies for Training Deep Neural Networks , 2009, J. Mach. Learn. Res..

[2]  Reza Ghaderi,et al.  Multi-class learning and error-correcting code sensitivity , 2000 .

[3]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[4]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[5]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[6]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[7]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.

[8]  Claudio Marrocco,et al.  Design of reject rules for ECOC classification systems , 2012, Pattern Recognit..

[9]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[10]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .

[11]  Stefano Marsili-Libelli,et al.  Adaptive mutation in genetic algorithms , 2000, Soft Comput..

[12]  R. Durrett.  Probability: Theory and Examples , 1993 .

[13]  Thomas G. Dietterich,et al.  Error-Correcting Output Codes: A General Method for Improving Multiclass Inductive Learning Programs , 1991, AAAI.

[14]  Ji Wu,et al.  Optimized weighted decoding for error-correcting output codes , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Ji Wu,et al.  Simple Deep Random Model Ensemble , 2013, ArXiv.

[16]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[17]  Elizabeth Tapia,et al.  Recursive ECOC classification , 2010, Pattern Recognit. Lett..

[18]  Venkatesan Guruswami,et al.  Multiclass learning, boosting, and error-correcting codes , 1999, COLT '99.

[19]  Rayid Ghani,et al.  Using Error-Correcting Codes for Text Classification , 2000, ICML.

[20]  Stephen V. Stehman,et al.  Selecting and interpreting measures of thematic classification accuracy , 1997 .

[21]  Adam L. Berger,et al.  ERROR-CORRECTING OUTPUT CODING FOR TEXT CLASSIFICATION , 1999 .

[22]  Terry Windeatt,et al.  Accuracy/Diversity and Ensemble MLP Classifier Design , 2006, IEEE Transactions on Neural Networks.

[23]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[24]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[25]  Carina Curto,et al.  Encoding Binary Neural Codes in Networks of Threshold-Linear Neurons , 2012, Neural Computation.

[26]  Robert P. W. Duin,et al.  A Fast Approach to Improve Classification Performance of ECOC Classification Systems , 2008, SSPR/SPR.

[27]  Yong Wang,et al.  The hybrid genetic algorithm with two local optimization strategies for traveling salesman problem , 2014, Comput. Ind. Eng..

[28]  Rita Cucchiara,et al.  Genetic algorithms for clustering in machine vision , 1998, Machine Vision and Applications.

[29]  Judy L. Walker,et al.  Combinatorial Neural Codes from a Mathematical Coding Theory Perspective , 2012, Neural Computation.

[30]  John J. Grefenstette,et al.  Genetic Algorithms for Changing Environments , 1992, PPSN.

[31]  John N. Tsitsiklis,et al.  Introduction to Probability , 2002 .

[32]  Giorgio Valentini,et al.  An Experimental Analysis of the Dependence Among Codeword Bit Errors in Ecoc Learning Machines , 2002, Neurocomputing.

[33]  Xin Yao,et al.  Making use of population information in evolutionary artificial neural networks , 1998, IEEE Trans. Syst. Man Cybern. Part B.