The Relationship Between the Multi-Collinearity and the Generalization Capability of the Neural Network Forecast Model

With the practical problem in application of the neural network in weather forecast, the effect of the learning matrix in neural network forecast model with the multi-collinearity on the generalization capability is researched. The results show, in the context of the same input knot number, whatever the smaller network or the network getting larger, there is few changes in simulation error both the neural network models without or with multi-collinearity, and the mean simulation errors for both of the two types model are very close, but the generalization capability of the neural network with multi-collinearity is obvious superior than that without multi-collinearity. Further more, it is to analyses the generalization capability for the two types of models in different training times from 5000 to 20000, the results also indicate that the multi-collinearity have the remarkable effect on decrease the forecast precision to the neural network forecast model

[1]  Andrew R. Barron,et al.  Approximation and estimation bounds for artificial neural networks , 2004, Machine Learning.

[2]  A. Barron.  Approximation and Estimation Bounds for Artificial Neural Networks , 1991, COLT '91.

[3]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[4]  G. Horvath,et al.  CMAC neural network with improved generalization property for system modeling , 2002, IMTC/2002. Proceedings of the 19th IEEE Instrumentation and Measurement Technology Conference (IEEE Cat. No.00CH37276).

[5]  G. Mirchandani,et al.  On hidden nodes for neural nets , 1989 .

[6]  Yan Wu,et al.  A new regularization learning method for improving generalization capability of neural network , 2002, Proceedings of the 4th World Congress on Intelligent Control and Automation (Cat. No.02EX527).

[7]  Jorge Lobo,et al.  A Policy Description Language , 1999, AAAI/IAAI.

[8]  Tamás Szabó,et al.  Improving the generalization capability of the binary CMAC , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[9]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.