A new hybrid methodology for cooperative-coevolutionary optimization of radial basis function networks

This paper presents a new multiobjective cooperative–coevolutive hybrid algorithm for the design of a Radial Basis Function Network (RBFN). This approach codifies a population of Radial Basis Functions (RBFs) (hidden neurons), which evolve by means of cooperation and competition to obtain a compact and accurate RBFN. To evaluate the significance of a given RBF in the whole network, three factors have been proposed: the basis function’s contribution to the network’s output, the error produced in the basis function radius, and the overlapping among RBFs. To achieve an RBFN composed of RBFs with proper values for these quality factors our algorithm follows a multiobjective approach in the selection process. In the design process, a Fuzzy Rule Based System (FRBS) is used to determine the possibility of applying operators to a certain RBF. As the time required by our evolutionary algorithm to converge is relatively small, it is possible to get a further improvement of the solution found by using a local minimization algorithm (for example, the Levenberg–Marquardt method). In this paper the results of applying our methodology to function approximation and time series prediction problems are also presented and compared with other alternatives proposed in the bibliography.

[1]  Héctor Pomares,et al.  A systematic approach to a self-generating fuzzy rule-table for function approximation , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Julio Ortega,et al.  Design of RBF Networks by Cooperative/Competitive Evolution of Units , 2001 .

[3]  Bart Kosko,et al.  Fuzzy function approximation with ellipsoidal rules , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[4]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[5]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[6]  Ernesto Tarantino,et al.  Optimizing Neural Networks for Time Series Prediction , 1999 .

[7]  Vladimir Cherkassky,et al.  Constrained topological mapping for nonparametric regression analysis , 1991, Neural Networks.

[8]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[9]  Roman Rosipal,et al.  Prediction of Chaotic Time-Series with a Resource-Allocating RBF Network , 1998, Neural Processing Letters.

[10]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[11]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[12]  Lakhmi C. Jain,et al.  Radial Basis Function Networks 2 , 2001 .

[13]  Juan Julián Merelo Guervós,et al.  Evolved RBF Networks for Time-Series Forecasting and Function Approximation , 2002, PPSN.

[14]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[15]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[16]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Evolutionary Radial Basis Functions for Credit Assessment , 2005, Applied Intelligence.

[17]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[18]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[19]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[20]  Rivera Rivas,et al.  Diseño y optimización de redes de funciones de base radial mediante técnicas bioinspiradas , 2003 .

[21]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[22]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[23]  Sheng Chen,et al.  Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks , 1999, IEEE Trans. Neural Networks.

[24]  Pedro Ángel Castillo Valdivieso,et al.  Optimización de perceptrones multicapa mediante algoritmos evolutivos , 2006 .

[25]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[26]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[27]  Ignacio Rojas,et al.  Co-evolutionary Algorithm for RBF by Self-Organizing Population of Neurons , 2003, IWANN.

[28]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[29]  Bruce A. Whitehead,et al.  Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction , 1996, IEEE Trans. Neural Networks.

[30]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[31]  Christian W. Dawson,et al.  A review of genetic algorithms applied to training radial basis function networks , 2004, Neural Computing & Applications.

[32]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[33]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[34]  Marco Tomassini,et al.  Soft computing - integrating evolutionary, neural, and fuzzy systems , 2001 .

[35]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[36]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[37]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Evolutionary Optimization of RBF Networks , 2001, Int. J. Neural Syst..

[38]  César Hervás-Martínez,et al.  Multi-objective cooperative coevolution of artificial neural networks (multi-objective cooperative networks) , 2002, Neural Networks.

[39]  J. Mendel.  Fuzzy logic systems for engineering: a tutorial , 1995, Proc. IEEE.

[40]  Jerome H. Friedman.  Multivariate adaptive regression splines (with discussion) , 1991 .

[41]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[42]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Evolutionary optimization of RBF networks , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.

[43]  Vladimir Cherkassky,et al.  Comparison of adaptive methods for function estimation from samples , 1996, IEEE Trans. Neural Networks.

[44]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[45]  Michael Conrad,et al.  Combining evolution with credit apportionment: A new learning algorithm for neural nets , 1994, Neural Networks.

[46]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..

[47]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[48]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[49]  H. Bersini,et al.  Using incremental learning algorithms in the search for minimal and effective fuzzy models , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[50]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[51]  Bernhard Sick,et al.  Evolutionary optimization of radial basis function classifiers for data mining applications , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[52]  Ignacio Rojas,et al.  Statistical Analysis of the Main Parameters in the Definition of Radial Bases Function Networks , 1997, IWANN.

[53]  P. A. Castillo.  Optimizaci on de perceptrones multicapa mediante algoritmos evolutivos , 2006 .

[54]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .