Fast and Efficient Training of RBF Networks

Radial basis function (RBF) networks are used in many applications, e.g. for pattern classification or nonlinear regression. Typically, either stochastic, iterative training algorithms (e.g. gradient-based or second-order techniques) or clustering methods in combination with a linear optimisation technique (e.g. c-means and singular value decomposition for a linear least-squares problem) are applied to find the parameters (centres, radii and weights) of an RBF network. This article points out the advantages of a combination of the two approaches and describes a modification of the standard c-means algorithm that leads to a linear least-squares problem for which solvability can be guaranteed. The first idea may lead to significant improvements concerning the training time as well as the approximation and generalisation properties of the networks. In the particular application problem investigated here (intrusion detection in computer networks), the overall training time could be reduced by about 29% and the error rate could be reduced by about 74%. The second idea rises the reliability of the training procedure at no additional costs (regarding both, run time and quality of results).

[1]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[2]  Man Wai Mak,et al.  Genetic evolution of radial basis function centers for pattern classification , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[3]  Kezhi Mao,et al.  RBF neural network center selection based on Fisher ratio class separability measure , 2002, IEEE Trans. Neural Networks.

[4]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[5]  A. G. Constantinides,et al.  An heuristic pattern correction scheme for GRNNs and its application to speech recognition , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[6]  J. C. Bennett,et al.  The application of artificial neural networks and standard statistical methods to SAR image classification , 1997, IGARSS'97. 1997 IEEE International Geoscience and Remote Sensing Symposium Proceedings. Remote Sensing - A Scientific Vision for Sustainable Development.

[7]  Bruce A. Whitehead,et al.  Evolving space-filling curves to distribute radial basis functions over an input space , 1994, IEEE Trans. Neural Networks.

[8]  Stephen A. Billings,et al.  Radial basis function network configuration using genetic algorithms , 1995, Neural Networks.

[9]  A. P. Dhawan,et al.  SSME parameter estimation using radial basis function neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[10]  Gene H. Golub,et al.  Matrix computations , 1983 .

[11]  Kevin Warwick,et al.  Robust initialisation of Gaussian radial basis function networks using partitioned k-means clustering , 1996 .

[12]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[13]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[14]  Tomaso A. Poggio,et al.  Extensions of a Theory of Networks for Approximation and Learning , 1990, NIPS.

[15]  Adrian J. Shepherd,et al.  Second-order methods for neural networks - fast and reliable training methods for multi-layer perceptrons , 1997, Perspectives in neural computing.

[16]  Adrian J. Shepherd,et al.  Second-Order Methods for Neural Networks , 1997 .

[17]  Bruce A. Whitehead,et al.  Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction , 1996, IEEE Trans. Neural Networks.

[18]  Michael R. Berthold,et al.  Ein Trainingsverfahren für Radial Basis Function Netzwerke mit dynamischer Selektion der Zentren und Adaption der Radii , 1994, Fuzzy Days.

[19]  Friedrich Gebhardt.  Fuzzy Logik - Theorie und Praxis , 1993, Künstliche Intell..

[20]  Donald E. Knuth,et al.  The art of computer programming, volume 3: (2nd ed.) sorting and searching , 1998 .

[21]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[22]  S. Shimoji,et al.  Data clustering with entropical scheduling , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[23]  Maria Cristina Felippetto de Castro,et al.  RBF neural networks with centers assignment via Karhunen-Loeve transform , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[24]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[25]  M. M. Brizzotti,et al.  The influence of clustering techniques in the RBF networks generalization , 1999 .

[26]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.