Weight Initialization of Feedforward Neural Networks by Means of Partial Least Squares

A method to set the weight initialization and the optimal number of hidden nodes of feedforward neural networks (FNN) based on the partial least squares (PLS) algorithm is developed. The combination of PLS and FNN method ensures that the outputs of neurons are in the active region and increases the rate of convergence. The performance of the FNN, PLS, and PLS-FNN are compared according to an example of customer satisfaction measurement with unknown relationship between the input and output data. The results show that the hybrid PLS-FNN has the smallest root mean square error and the highest imitating precision. It substantially provides the good initial weights, improves the training performance and efficiently achieves an optimal solution

[1]  Thierry Denoeux,et al.  Initializing back propagation networks with prototypes , 1993, Neural Networks.

[2]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[3]  Ronald D. Snee,et al.  Validation of Regression Models: Methods and Examples , 1977 .

[4]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[5]  Sandro Ridella,et al.  Statistically controlled activation weight initialization (SCAWI) , 1992, IEEE Trans. Neural Networks.

[6]  S. Wold,et al.  PLS-regression: a basic tool of chemometrics , 2001 .

[7]  K. Kristensen,et al.  Customer satisfaction measurement at Post Denmark: Results of application of the European Customer Satisfaction Index Methodology , 2000 .

[8]  V. Rao Vemuri,et al.  Artificial Neural Networks - Forecasting Time Series , 1993 .

[9]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[10]  K. Kristensen,et al.  The drivers of customer satisfaction and loyalty: Cross-industry findings from Denmark , 2000 .

[11]  Metin Akay.  Artificial Neural Networks for Spectroscopic Signal Measurement , 2000 .

[12]  J. F. Shepanski.  Fast learning in artificial neural systems: multilayer perceptron training using optimal estimation , 1988, IEEE 1988 International Conference on Neural Networks.

[13]  Hisashi Shimodaira,et al.  A weight value initialization method for improving learning performance of the backpropagation algorithm in neural networks , 1994, Proceedings Sixth International Conference on Tools with Artificial Intelligence. TAI 94.

[14]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[15]  K. Kristensen,et al.  The relationship between customer satisfaction and loyalty: Cross-industry differences , 2000 .

[16]  James L. McClelland.  Parallel Distributed Processing , 2005 .

[17]  Alberto Tesi,et al.  On the Problem of Local Minima in Backpropagation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Timothy Masters,et al.  Practical neural network recipes in C , 1993 .