Pilot Study on the Localized Generalization Error Model for Single Layer Perceptron Neural Network

We had developed the localized generalization error model for supervised learning with minimization of mean square error. In this work, we extend the error model to single layer perceptron neural network (SLPNN). For a trained SLPNN and a given training dataset, the proposed error model bounds above the error for unseen samples which are similar to the training samples. This pilot study is the important first step of investigating localized generalization error models for multilayer perceptron neural networks and support vector machines with sigmoid kernel functions. The characteristics of the error model for SLPNN and how to compare SLPNNs' generalization capabilities using the error model are also discussed in this paper

[1]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[2]  Sumio Watanabe,et al.  Algebraic Analysis for Nonidentifiable Learning Machines , 2001, Neural Computation.

[3]  Trevor Hastie,et al.  The elements of statistical learning. 2001 , 2001 .

[4]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[5]  Daniel S. Yeung,et al.  Localized Generalization Error of Gaussian-based Classifiers and Visualization of Decision Boundaries , 2006, Soft Comput..

[6]  Shun-ichi Amari,et al.  Improving Generalization Performance of Natural Gradient Learning Using Optimized Regularization by NIC , 2004, Neural Computation.

[7]  D. S. Yeung,et al.  Localized generalization error and its application to RBFNN training , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[8]  Simon Haykin,et al.  Neural networks , 1994 .

[9]  H. Akaike.  A Bayesian analysis of the minimum AIC procedure , 1978 .

[10]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[11]  Vladimir Cherkassky,et al.  Model complexity control for regression using VC generalization bounds , 1999, IEEE Trans. Neural Networks.

[12]  Nikhil R. Pal,et al.  A novel training scheme for multilayered perceptrons to realize proper generalization and incremental learning , 2003, IEEE Trans. Neural Networks.

[13]  Lakhmi C. Jain,et al.  Industrial Applications of Neural Networks , 1998 .

[14]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.