The Design of RBF Neural Networks for Solving Overfitting Problem

One of the biggest problems in designing or training RBF neural networks are the overfitting problem. The traditional design of RBF neural networks may be pursued in a variety of ways. In this paper, we present a method for the design of RBF networks to solve overfitting problem. For a practical application, frequency information is usually available for designing RBF networks by frequency domain analysis, which has a sound mathematical basis. We try to include the frequency information into the design of RBF networks, which achieve the task of approximated a function in certain frequency range and have the property of structural risk minimization. After the structure of designed network is determined, the linear weights of the output layer are the only set of adjustable parameters. The approach of design is verified by approximation cases

[1]  Marimuthu Palaniswami,et al.  Effects of moving the centers in an RBF network , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[2]  L. Goddard.  Linear Differential Operators , 1962, Nature.

[3]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[4]  Tianping Chen,et al.  Approximation capability to functions of several variables, nonlinear functionals and operators by radial basis function neural networks , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[5]  Irwin W. Sandberg.  Gaussian radial basis functions and the approximation of input-output maps , 2003, Int. J. Circuit Theory Appl..

[6]  Marimuthu Palaniswami,et al.  Effects of moving the center's in an RBF network , 2002, IEEE Trans. Neural Networks.

[7]  Ying-Hua Lu,et al.  Center selection for RBF neural network in prediction of nonlinear time series , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[8]  Irwin W. Sandberg,et al.  Gaussian radial basis functions and the approximation of input-output maps , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[9]  James J. Carroll,et al.  Approximation of nonlinear systems with radial basis function neural networks , 2001, IEEE Trans. Neural Networks.

[10]  Kumpati S. Narendra,et al.  Identification and control , 1998 .

[11]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[12]  P. Goldbart,et al.  Linear differential operators , 1967 .

[13]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[14]  A. Kirsch.  An Introduction to the Mathematical Theory of Inverse Problems , 1996, Applied Mathematical Sciences.