Regularization in the Selection of Radial Basis Function Centers

Subset selection and regularization are two well-known techniques that can improve the generalization performance of nonparametric linear regression estimators, such as radial basis function networks. This paper examines regularized forward selection (RFS)a combination of forward subset selection and zero-order regularization. An efficient implementation of RFS into which either delete-1 or generalized cross-validation can be incorporated and a reestimation formula for the regularization parameter are also discussed. Simulation studies are presented that demonstrate improved generalization performance due to regularization in the forward selection of radial basis function centers.

[1]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .

[2]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[3]  R. R. Hocking.  Developments in Linear Regression Methodology: 1959–l982 , 1983 .

[4]  R. R. Hocking.  Developments in linear regression methodology: 1959-1982 , 1983 .

[5]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[6]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[7]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[8]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[9]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[10]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[11]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[12]  A. Barron,et al.  Discussion: Multivariate Adaptive Regression Splines , 1991 .

[13]  Chong Gu,et al.  Minimizing GCV/GML Scores with Multiple Smoothing Parameters via the Newton Method , 1991, SIAM J. Sci. Comput..

[14]  Chris Bishop,et al.  Improving the Generalization Properties of Radial Basis Function Neural Networks , 1991, Neural Computation.

[15]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[16]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[17]  Bernd Fritzke.  Supervised Learning with Growing Cell Structures , 1993, NIPS.

[18]  Visakan Kadirkamanathan,et al.  A Function Estimation Approach to Sequential Learning with Neural Networks , 1993, Neural Computation.

[19]  Mark J. L. Orr.  Regularised Centre Recruitment in Radial Basis Function Networks , 1993 .

[20]  Sheng Chen,et al.  Regularized orthogonal least squares algorithm for constructing radial basis function networks , 1996 .