Developing FFNN applications using cross-validated validation training

In this paper, we present a novel, effective, and reliable training technique for feed–forward neural networks (FFNN). We call it cross–validated validation training (CVVT) since it combines statistical cross–validation with the validation training technique used in FFNNs. CVVT improves the generalisation estimation of validation training, enabling reliable comparison and selection of network architectures. Since it utilises validation training, CVVT also preserves the generalisation performance of FFNNs with excess weights. These benefits are demonstrated using statistical analysis of real–life results from a bake inspection system. Contrary to previous work, we found that significant excess weights may actually deteriorate the generalisation preserving ability of validation

[1]  J. Utans,et al.  Selecting neural network architectures via the prediction risk: application to corporate bond rating prediction , 1991, Proceedings First International Conference on Artificial Intelligence Applications on Wall Street.

[2]  John W. Tukey,et al.  Data Analysis and Regression: A Second Course in Statistics , 1977 .

[3]  Bernard Ostle,et al.  Statistics in Research: Basic Concepts and Techniques for Research Workers , 1990 .

[4]  Warren S. Sarle,et al.  Stopped Training and Other Remedies for Overfitting , 1995 .

[5]  Arthur Flexer,et al.  Statistical evaluation of neural networks experiments: Minimum requirements and current practice , 1994 .

[6]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[7]  Hervé Bourlard,et al.  Generalization and Parameter Estimation in Feedforward Netws: Some Experiments , 1989, NIPS.

[8]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[9]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[10]  Casimir A. Kulikowski,et al.  Computer Systems That Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning and Expert Systems , 1990 .

[11]  L. Ljung,et al.  Overtraining, Regularization, and Searching for Minimum in Neural Networks , 1992 .

[12]  C. Christensen.  Effects of color on aroma, flavor and texture judgments of foods , 1983 .

[13]  Frederick Mosteller,et al.  Data Analysis and Regression , 1978 .