Nonlinear Function Learning Using Radial Basis Function Networks: Convergence and Rates

We apply normalized RBF networks to the problem of learning nonlinear regression functions. The parameters of the networks are learned by empirical risk minimization and complexity regularization. We study convergence of the RBF networks for various radial kernels as the number of training samples increases. The rates of convergence are also examined.

[1]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[2]  András Faragó,et al.  Strong universal consistency of neural network classifiers , 1993, IEEE Trans. Inf. Theory.

[3]  Kurt Hornik,et al.  FEED FORWARD NETWORKS ARE UNIVERSAL APPROXIMATORS , 1989 .

[4]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[5]  Adam Krzyzak,et al.  Nonparametric regression estimation by normalized radial basis function networks , 2003, IEEE Transactions on Information Theory.

[6]  Gábor Lugosi,et al.  Nonparametric estimation via empirical risk minimization , 1995, IEEE Trans. Inf. Theory.

[7]  Federico Girosi,et al.  Regularization Theory, Radial Basis Functions and Networks , 1994 .

[8]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[9]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[10]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[11]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[12]  A. Krzyżak,et al.  Convergence and rates of convergence of radial basis functions networks in function learning , 2001 .

[13]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[14]  J. Duchon.  Sur l’erreur d’interpolation des fonctions de plusieurs variables par les $D^m$-splines , 1978 .

[15]  Adam Krzyzak,et al.  Radial Basis Function Networks and Complexity Regularization in Function Learning , 2022 .

[16]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[17]  A. Barron.  Approximation and Estimation Bounds for Artificial Neural Networks , 1991, COLT '91.

[18]  中澤 真,et al.  Devroye, L., Gyorfi, L. and Lugosi, G. : A Probabilistic Theory of Pattern Recognition, Springer (1996). , 1997 .

[19]  Richard J. Mammone,et al.  Artificial neural networks for speech and vision , 1994 .

[20]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[21]  David Haussler,et al.  Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications , 1992, Inf. Comput..

[22]  Peter L. Bartlett,et al.  Learning in Neural Networks: Theoretical Foundations , 1999 .

[23]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[24]  D. Pollard.  Convergence of stochastic processes , 1984 .

[25]  Adam Krzyzak,et al.  On radial basis function nets and kernel regression: Statistical consistency, convergence rates, and receptive field size , 1994, Neural Networks.

[26]  Harry Wechsler,et al.  From Statistics to Neural Networks , 1994, NATO ASI Series.

[27]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[28]  Robert Shorten,et al.  Side effects of Normalising Radial Basis Function Networks , 1996, Int. J. Neural Syst..

[29]  Vladimir Vapnik,et al.  Estimation of Dependences Based on Empirical Data: Empirical Inference Science (Information Science and Statistics) , 2006 .

[30]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[31]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[32]  Adam Krzyzak,et al.  Nonparametric estimation and classification using radial basis function nets and empirical risk minimization , 1996, IEEE Trans. Neural Networks.

[33]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.