A Comparison of Some Error Estimates for Neural Network Models

We discuss a number of methods for estimating the standard error of predicted values from a multilayer perceptron. These methods include the delta method based on the Hessian, bootstrap estimators, and the sandwich estimator. The methods are described and compared in a number of examples. We find that the bootstrap methods perform best, partly because they capture variability due to the choice of starting weights.

[1]  Franklin A. Graybill,et al.  Introduction to The theory , 1974 .

[2]  J. Nash.  Compact Numerical Methods for Computers , 2018 .

[3]  J. Kent.  Robust properties of likelihood ratio tests , 1982 .

[4]  R. Lippmann.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[5]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[6]  Geoffrey E. Hinton.  20 – CONNECTIONIST LEARNING PROCEDURES1 , 1990 .

[7]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[8]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[9]  Lyle H. Ungar,et al.  A NEURAL NETWORK ARCHITECTURE THAT COMPUTES ITS OWN RELIABILITY , 1992 .

[10]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[11]  A. Weigend,et al.  Estimating the mean and variance of the target probability distribution , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[12]  Blake LeBaron,et al.  Evaluating Neural Network Predictors by Bootstrapping , 1994 .

[13]  Wray L. Buntine,et al.  Computing second derivatives in feed-forward networks: a review , 1994, IEEE Trans. Neural Networks.

[14]  Halbert White,et al.  Bootstrapping Confidence Intervals for Clinical Input Variable Effects in a Network Trained to Identify the Presence of Acute Myocardial Infarction , 1995, Neural Computation.

[15]  L. K. Hansen,et al.  Generalization: The Hidden Agenda of Learning , 1997, IEEE Signal Process. Mag..

[16]  C. Kiparissides,et al.  Inferential Estimation of Polymer Quality Using Stacked Neural Networks , 1997 .

[17]  Paul Lajbcygier,et al.  Improved option pricing using bootstrap methods , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[18]  Paul Lajbcygier,et al.  Improved Option Pricing Using Artificial Neural Networks and Bootstrap Methods , 1997, Int. J. Neural Syst..

[19]  Boualem Boashash,et al.  The bootstrap and its application in signal processing , 1998, IEEE Signal Process. Mag..

[20]  Blake LeBaron,et al.  A Bootstrap Evaluation of the Effect of Data Splitting on Financial Time Series , 1996, IEEE Trans. Neural Networks.