A review of genetic algorithms applied to training radial basis function networks

The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.

[1]  David E. Goldberg,et al.  Real-coded Genetic Algorithms, Virtual Alphabets, and Blocking , 1991, Complex Syst..

[2]  Vasant Honavar,et al.  Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature , 1995 .

[3]  Ali M. S. Zalzala,et al.  Evolving hybrid RBF-MLP networks using combined genetic/unsupervised/supervised learning , 1998 .

[4]  S. N. Sivanandam,et al.  Desing of a soft computing hybrid model classifier for data mining applications , 2001 .

[5]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[6]  Gerrit Kateman,et al.  Towards Solving Subset Selection Problems with the Aid of the Genetic Algorithm , 1992, PPSN.

[7]  W. M. Jenkins,et al.  Genetic Algorithms and Neural Networks , 1999, Neural Networks in the Analysis and Design of Structures.

[8]  Stephen A. Billings,et al.  Radial basis function network configuration using genetic algorithms , 1995, Neural Networks.

[9]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .

[10]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[11]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[12]  Manfred Morari,et al.  Local Training for Radial Basis Function Networks: Towards Solving the Hidden Unit Problem , 1991, 1991 American Control Conference.

[13]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[14]  Bull,et al.  An Overview of Genetic Algorithms: Part 2, Research Topics , 1993 .

[15]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[16]  Christophe Giraud-Carrier,et al.  GA-RBF: A Self-Optimising RBF Network , 1997, ICANNGA.

[17]  R. Hecht-Nielsen.  ON THE ALGEBRAIC STRUCTURE OF FEEDFORWARD NETWORK WEIGHT SPACES , 1990 .

[18]  Brian Carse,et al.  Tackling the "Curse of Dimensionality" of Radial Basis Functional Neural Networks Using a Genetic Algorithm , 1996, PPSN.

[19]  Alaa F. Sheta,et al.  Time-series forecasting using GA-tuned radial basis functions , 2001, Inf. Sci..

[20]  Howell Tong,et al.  Non-Linear Time Series , 1990 .

[21]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[22]  D. Fogel.  Evolutionary algorithms in theory and practice , 1997, Complex..

[23]  David Beasley,et al.  An overview of genetic algorithms: Part 1 , 1993 .

[24]  Timothy Masters,et al.  Probabilistic Neural Networks , 1993 .

[25]  S. N. Sivanandam,et al.  Design of a hybrid model classifier for data mining applications , 2000, SPIE Defense + Commercial Sensing.

[26]  Sheng Chen,et al.  Practical identification of NARMAX models using radial basis functions , 1990 .

[27]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[28]  Ludmila I. Kuncheva,et al.  Initializing of an RBF network by a genetic algorithm , 1997, Neurocomputing.

[29]  Alistair Munro,et al.  Evolving fuzzy rule based controllers using genetic algorithms , 1996, Fuzzy Sets Syst..

[30]  Sukhan Lee,et al.  A Gaussian potential function network with hierarchically self-organizing learning , 1991, Neural Networks.

[31]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.

[32]  S. Aiguo,et al.  Evolving Gaussian RBF network for nonlinear time series modelling and prediction , 1998 .

[33]  David Coley,et al.  Introduction to Genetic Algorithms for Scientists and Engineers , 1999 .

[34]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[35]  Larry R. Medsker,et al.  Genetic Algorithms and Neural Networks , 1995 .

[36]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[37]  Colin R. Reeves,et al.  Selection of Training Data for Neural Networks by a Genetic Algorithm , 1998, PPSN.

[38]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[39]  Bruce A. Whitehead,et al.  Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction , 1996, IEEE Trans. Neural Networks.

[40]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[41]  Roman Neruda.  Functional Equivalence and Genetic Learning of RBF Networks , 1995, ICANNGA.

[42]  James T. Kwok,et al.  Constructive algorithms for structure learning in feedforward neural networks for regression problems , 1997, IEEE Trans. Neural Networks.

[43]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[44]  S. Chen,et al.  A two-layer learning method for radial basis function networks using combined genetic and regularised OLS algorithms , 1995 .

[45]  Jean-Marc Vesin,et al.  Model selection using a simplex reproduction genetic algorithm , 1999, Signal Process..

[46]  B. K. Natarajan,et al.  Genetic Evolution of Radial Basis Function Coverage Using Orthogonal Niches , 1996 .

[47]  Brian Carse,et al.  Fast Evolutionary Learning of Minimal Radial Basis Function Neural Networks Using a Genetic Algorithm , 1996, Evolutionary Computing, AISB Workshop.

[48]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[49]  B. G. Batchelor,et al.  Method for location of clusters of patterns to initialise a learning machine , 1969 .

[50]  B. Carse,et al.  Evolving radial basis function neural networks using a genetic algorithm , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[51]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (3rd ed.) , 1996 .

[52]  S. A. Sergeev,et al.  Genetic algorithm for training dynamical object emulator based on RBF neural network , 1998 .

[53]  T. Kohonen.  Self-Organized Formation of Correct Feature Maps , 1982 .

[54]  Lakhmi C. Jain,et al.  Neural Network Training Using Genetic Algorithms , 1996 .

[55]  Mahesan Niranjan,et al.  Neural networks and radial basis functions in classifying static speech patterns , 1990 .

[56]  Sheng Chen,et al.  Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks , 1999, IEEE Trans. Neural Networks.

[57]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[58]  Chuen-Tsai Sun,et al.  Functional equivalence between radial basis function networks and fuzzy inference systems , 1993, IEEE Trans. Neural Networks.

[59]  J. Watton,et al.  Dynamics modelling of fluid power systems applying a global error descent algorithm to a self-organising Radial Basis Function network , 1998 .

[60]  H. Leung,et al.  Chaotic radar signal processing over the sea , 1993 .

[61]  Zhiye Zhao,et al.  Design of structural modular neural networks with genetic algorithm , 2003 .

[62]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[63]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[64]  J. Rissanen.  Stochastic Complexity in Statistical Inquiry Theory , 1989 .

[65]  Xin Yao,et al.  Evolutionary design of artificial neural networks with different nodes , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[66]  Peter J. B. Hancock,et al.  Genetic algorithms and permutation problems: a comparison of recombination operators for neural net structure specification , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[67]  Bruce A. Whitehead,et al.  Evolving space-filling curves to distribute radial basis functions over an input space , 1994, IEEE Trans. Neural Networks.

[68]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[69]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[70]  Yu. V. Orlov,et al.  Benchmarking of Different Modifications of the Cascade Correlation Algorithm , 1998 .

[71]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[72]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[73]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[74]  Henry Leung,et al.  Detection of small objects in clutter using a GA-RBF neural network , 2002 .

[75]  A. S. Farag,et al.  Combined genetic algorithms and neural-network approach for power-system transient stability evaluation , 1999 .

[76]  Narasimhan Sundararajan,et al.  Radial Basis Function Neural Networks With Sequential Learning: Mran and Its Applications , 1999 .

[77]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[78]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[79]  T. Kohonen.  Self-organized formation of topology correct feature maps , 1982 .

[80]  Christophe Giraud-Carrier,et al.  Evolving fuzzy prototypes for efficient data clustering , 1997 .

[81]  Kalyanmoy Deb,et al.  An Investigation of Niche and Species Formation in Genetic Function Optimization , 1989, ICGA.

[82]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[83]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.