Adaptive learning algorithm for RBF Neural Networks in kernel spaces

An adaptive learning algorithm for Radial Basis Functions Neural Networks, RBFNNs, is provided. In recent years, RBFs have been subject to extensive areas of interests. But the setting up of RBFs in a network architecture can be time consuming, computationally deficient and unstable. Thus we have developed an efficient adaptive algorithm in a feedforward neural architecture in which the hidden neurons are Newton bases. These bases are derived from the RBFs on optimal center points. Unlike the RBFs, the Newton bases are stable and orthonormal in the kernel space. The algorithm is implemented on variably scaled RBFs that simultaneously adjust the shape parameter. The procedure of the training is computationally efficient, accurate and stable in the kernel spaces specially for noisy data set. Numerical modeling for time series shows that our proposed algorithm has very promising performance in compare to some recent approaches.

[1]  Lipo Wang,et al.  Rule extraction by genetic algorithms based on a simplified RBF neural network , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[2]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[3]  Robert Schaback,et al.  Bases for kernel-based spaces , 2011, J. Comput. Appl. Math..

[4]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[5]  Dietmar P. F. Möller,et al.  An efficient learning method for RBF Neural Networks , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[6]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[7]  Nicolaos B. Karayiannis,et al.  Reformulated radial basis neural networks trained by gradient descent , 1999, IEEE Trans. Neural Networks.

[8]  Robert Schaback,et al.  Stability of kernel-based interpolation , 2010, Adv. Comput. Math..

[9]  Sven Behnke,et al.  Competitive neural trees for pattern classification , 1998, IEEE Trans. Neural Networks.

[10]  Shmuel Rippa,et al.  An algorithm for selecting a good value for the parameter c in radial basis function interpolation , 1999, Adv. Comput. Math..

[11]  Jürgen Schmidhuber,et al.  Flat Minima , 1997, Neural Computation.

[12]  R. Schaback,et al.  Numerical Techniques Based on Radial Basis Functions , 2000 .

[13]  John Mark,et al.  Introduction to radial basis function networks , 1996 .

[14]  Sander M. Bohte,et al.  Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks , 2002, IEEE Trans. Neural Networks.

[15]  Gregory E. Fasshauer,et al.  Meshfree Approximation Methods with Matlab , 2007, Interdisciplinary Mathematical Sciences.

[16]  R. Cozzi.  Interpolation with Variably Scaled Kernels , 2013 .

[17]  A. U.S.,et al.  Stable Computation of Multiquadric Interpolants for All Values of the Shape Parameter , 2003 .

[18]  Christopher M. Bishop,et al.  Current address: Microsoft Research, , 2022 .

[19]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[20]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[21]  Lipo Wang,et al.  Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[22]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[23]  Bernd Fritzke,et al.  Fast learning with incremental RBF networks , 1994, Neural Processing Letters.

[24]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[25]  Marimuthu Palaniswami,et al.  Effects of moving the center's in an RBF network , 2002, IEEE Trans. Neural Networks.

[26]  George W. Irwin,et al.  A Novel Continuous Forward Algorithm for RBF Neural Modelling , 2007, IEEE Transactions on Automatic Control.

[27]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[28]  M. Urner.  Scattered Data Approximation , 2016 .

[29]  Juan Julián Merelo Guervós,et al.  Evolving RBF neural networks for time-series forecasting with EvRBF , 2004, Inf. Sci..

[30]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[31]  Jean Meinguet,et al.  Optimal approximation and error bounds in seminormed spaces , 1967 .

[32]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.