Architecture optimization of radial basis function networks with a combination of hard- and soft-computing techniques

Feature selection and architecture optimization are two key tasks in many neural network applications. Appropriate input features must be selected from a given (and often large) set of possible features and architecture parameters of the network such as the number of hidden neurons or training parameters must be adapted with respect to the selected features and a data set given. This article describes an evolutionary algorithm (EA) that performs the two tasks simultaneously for radial basis functions (RBF) networks applied to classification problems. In order to reduce the optimization effort significantly these soft-computing techniques are focused with various hard-computing techniques (e.g., clustering, solution of a linear least-squares problem, local search). The feasibility and the benefits of the approach are demonstrated by means of a data mining and knowledge discovery problem in the area of customer relationship management. The algorithm, however, is independent from specific applications such that the ideas and solutions may easily be transferred to other applications and even other neural network paradigms.

[1]  Bruce A. Whitehead,et al.  Genetic evolution of radial basis function coverage using orthogonal niches , 1996, IEEE Trans. Neural Networks.

[2]  Kalyanmoy Deb,et al.  Understanding Interactions among Genetic Algorithm Parameters , 1998, FOGA.

[3]  De-shuang Huang,et al.  The structure optimization of radial basis probabilistic neural networks based on genetic algorithms , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[4]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Evolutionary Optimization of RBF Networks , 2001, Int. J. Neural Syst..

[5]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[6]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[7]  Lipo Wang,et al.  A GA-based RBF classifier with class-dependent features , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[8]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[9]  Bernhard Sick,et al.  Evolutionary optimization of radial basis function networks for intrusion detection , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[11]  Man Wai Mak,et al.  Genetic evolution of radial basis function centers for pattern classification , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[12]  Lakhmi C. Jain,et al.  Radial basis function networks 1: recent developments in theory and applications , 2001 .

[13]  Wendy Gersten,et al.  Evolutionary Optimisation of RBF Network Architectures in a Direct Marketing Application , 2003, ICANN.

[14]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[15]  Bruce A. Whitehead,et al.  Evolving space-filling curves to distribute radial basis functions over an input space , 1994, IEEE Trans. Neural Networks.

[16]  Stephen A. Billings,et al.  Radial basis function network configuration using genetic algorithms , 1995, Neural Networks.

[17]  Henry Leung,et al.  Detection of small objects in clutter using a GA-RBF neural network , 2002 .

[18]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[19]  Wendy Gersten,et al.  LIFT - A Leading Measure for Predictive Modeling , 2002, ICMLA.

[20]  David G. Stork,et al.  Pattern Classification , 1973 .

[21]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[22]  B. Sick,et al.  A strategy for an efficient training of radial basis function networks for classification applications , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[23]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[24]  Xin Yao,et al.  Evolutionary Artificial Neural Networks , 1993, Int. J. Neural Syst..

[25]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[26]  Lipo Wang,et al.  Rule extraction from an RBF classifier based on class-dependent features , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[27]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.