Estimations of error bounds for neural-network function approximators

Neural networks are being increasingly used for problems involving function approximation. However, a key limitation of neural methods is the lack of a measure of how much confidence can be placed in output estimates. In the last few years many authors have addressed this shortcoming from various angles, focusing primarily on predicting output bounds as a function of the trained network's characteristics, typically as defined by the Hessian matrix. In this paper the problem of the effect of errors or noise in the presented, input, vector is examined and a method based on perturbation analysis of determining output bounds based on both the error in the input vector and the imperfections in the weight values after training is presented and demonstrated.

[1]  Narendra Ahuja,et al.  Motion and Structure From Two Perspective Views: Algorithms, Error Analysis, and Error Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Stephen J. Roberts,et al.  Supervised and unsupervised learning in radial basis function classifiers , 1994 .

[3]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[4]  George Chryssolouris,et al.  Confidence interval prediction for neural network models , 1996, IEEE Trans. Neural Networks.

[5]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[6]  T. K. Leen.  Learning Local Error Bars for Nonlinear Regression , 1995 .

[7]  Adam Krzyzak,et al.  Nonparametric estimation and classification using radial basis function nets and empirical risk minimization , 1996, IEEE Trans. Neural Networks.

[8]  Bernard Widrow,et al.  Sensitivity of feedforward neural networks to weight errors , 1990, IEEE Trans. Neural Networks.

[9]  Naftali Tishby,et al.  Consistent inference of probabilities in layered networks: predictions and generalizations , 1989, International 1989 Joint Conference on Neural Networks.

[10]  Christopher M. Bishop,et al.  Novelty detection and neural network validation , 1994 .

[11]  James D. Keeler,et al.  Predicting the Future: Advantages of Semilocal Units , 1991, Neural Computation.

[12]  Christopher M. Bishop,et al.  Bayesian Inference of Noise Levels in Regression , 1996, ICANN.

[13]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[14]  Robert Tibshirani,et al.  A Comparison of Some Error Estimates for Neural Network Models , 1996, Neural Computation.

[15]  M. Brady,et al.  Rejecting outliers and estimating errors in an orthogonal-regression framework , 1995, Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences.

[16]  Yann LeCun,et al.  Transforming Neural-Net Output Levels to Probability Distributions , 1990, NIPS.

[17]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[18]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[19]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[20]  D.R. Hush,et al.  Progress in supervised neural networks , 1993, IEEE Signal Processing Magazine.

[21]  P. M. Williams,et al.  Using Neural Networks to Model Conditional Multivariate Densities , 1996, Neural Computation.

[22]  Lionel Tarassenko,et al.  Neural Networks for Mobile Robot Localisation using Infra-Red Range Sensing , 1999, Neural Computing & Applications.

[23]  Adam Krzyzak,et al.  On radial basis function nets and kernel regression: Statistical consistency, convergence rates, and receptive field size , 1994, Neural Networks.

[24]  C. Bishop.  Mixture density networks , 1994 .

[25]  David H. Wolpert,et al.  Bayesian Backpropagation Over I-O Functions Rather Than Weights , 1993, NIPS.