Pattern Classification by Evolutionary RBF Networks Ensemble Based on Multi-objective Optimization

In this paper, evolutionary multi-objective selection method of RBF networks structure and its application to the ensemble learning is considered. The candidates of RBF network structure are encoded into the chromosomes in GAs. Then, they evolve toward Pareto-optimal front defined by several objective functions concerning with model accuracy, model complexity and model smoothness. RBF network ensemble is constructed of the obtained Pareto-optimal models since such models are diverse. This method is applied to the pattern classification problem. Experiments on the benchmark problem demonstrate that the proposed method has comparable generalization ability to conventional ensemble methods.

[1]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[2]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[3]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[4]  K. Uosaki,et al.  Pareto RBF networks based on multiobjective evolutionary computation , 2004, SICE 2004 Annual Conference.

[5]  Isao Ono,et al.  A Real Coded Genetic Algorithm for Function Optimization Using Unimodal Normal Distributed Crossover , 1997, ICGA.

[6]  Yunfei Bai,et al.  Genetic algorithm based self-growing training for RBF neural networks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[7]  Gary G. Yen,et al.  Hierarchical Rank Density Genetic Algorithm for Radial-Basis Function Neural Network Design , 2003, Int. J. Comput. Intell. Appl..

[8]  Peter J. Fleming,et al.  Multiobjective Genetic Programming: A Nonlinear System Identification Application , 1997 .

[9]  Shigenobu Kobayashi,et al.  A Real-Coded Genetic Algorithm for Function Optimization Using the Unimodal Normal Distribution Crossover , 1999 .

[10]  Hussein A. Abbass,et al.  Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[11]  Bernhard Sendhoff,et al.  Neural network regularization and ensembling using multi-objective evolutionary algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[12]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[13]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[14]  De-shuang Huang,et al.  The structure optimization of radial basis probabilistic neural networks based on genetic algorithms , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[15]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[16]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[17]  D. Obradovic,et al.  Combining Artificial Neural Nets , 1999, Perspectives in Neural Computing.

[18]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[19]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[20]  Hussein A. Abbass,et al.  A Memetic Pareto Evolutionary Approach to Artificial Neural Networks , 2001, Australian Joint Conference on Artificial Intelligence.