On the Generalization Ability of Neural Network Classifiers

This correspondence presents a method for evaluation of artificial neural network (ANN) classifiers. In order to find the performance of the network over all possible input ranges, a probabilistic input model is defined. The expected error of the output over this input range is taken as a measure of generalization ability. Two essential elements for carrying out the proposed evaluation technique are estimation of the input probability density and numerical integration. A nonparametric method, which depends on the nearest M neighbors, is used to locally estimate the distribution around each training pattern. An orthogonalization procedure is utilized to determine the covariance matrices of local densities. A Monte Carlo method is used to perform the numerical integration. The proposed evaluation technique has been used to investigate the generalization ability of back propagation (BP), radial basis function (RBF) and probabilistic neural network (PNN) classifiers for three test problems. >

[1]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[2]  Gene H. Golub,et al.  Matrix computations , 1983 .

[3]  Roy Billinton,et al.  Reliability Evaluation of Engineering Systems , 1983 .

[4]  Keinosuke Fukunaga,et al.  Estimation of Classifier Performance , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Simon Haykin,et al.  A multi-layer neural network classifier for radar clutter , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[6]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[7]  S. Yoshimoto.  A study on artificial neural network generalization capability , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[8]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.

[9]  John M. Libert,et al.  Classifying seismic signals via RCE neural network , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[10]  Donald F. Specht,et al.  Probabilistic neural networks and the polynomial Adaline as complementary techniques for classification , 1990, IEEE Trans. Neural Networks.

[11]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[12]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[13]  Simon Haykin,et al.  Radial basis function classification of impulse radar waveforms , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[14]  Donald F. Specht,et al.  Generation of Polynomial Discriminant Functions for Pattern Recognition , 1967, IEEE Trans. Electron. Comput..

[15]  Alexander H. Waibel,et al.  Speaker-independent phoneme recognition on TIMIT database using integrated time-delay neural networks (TDNNs) , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[16]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[17]  William S. Meisel,et al.  Computer-oriented approaches to pattern recognition , 1972 .

[18]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[19]  C. Micchelli.  Interpolation of scattered data: Distance matrices and conditionally positive definite functions , 1986 .

[20]  Y.-H. Yu,et al.  Descending epsilon in back-propagation: a technique for better generalization , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[21]  Mohamad T. Musavi,et al.  On the implementation of RBF technique in neural networks , 1991, ANNA '91.

[22]  Richard Lippmann,et al.  Neural Network Classifiers Estimate Bayesian a posteriori Probabilities , 1991, Neural Computation.

[23]  V. A. Epanechnikov.  Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[24]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[25]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[26]  Mohamad T. Musavi,et al.  A probabilistic model for evaluation of neural network classifiers , 1992, Pattern Recognit..

[27]  Todd K. Leen,et al.  Hebbian feature discovery improves classifier efficiency , 1990, 1990 IJCNN International Joint Conference on Neural Networks.