论文信息 - Uncertain Relation Suited to Overfitting of BP Neural Network

Uncertain Relation Suited to Overfitting of BP Neural Network

A general uncertainty relation between the change of weighted value which represents learning ability and the discrimination error of unlearning sample sets which represents generalization ability is revealed in the modeling of back propagation (BP) neural network. Tests of numerical simulation for multitype of complicated functions are carried out to determine the value distribution (1×10 −5 ~5×10 −4 ) of overfitting parameter in the uncertainty relation. Based on the uncertainty relation, the overfitting in the training process of given sample sets using BP neural network can be judged.

Jiang Chunhua | Li Zuoyong | Ren Jiping | Yang Haomiao

[1] David H. Wolpert,et al. A Mathematical Theory of Generalization: Part II , 1990, Complex Syst..

[2] Esther Levin,et al. A statistical approach to learning and generalization in layered neural networks , 1989, Proc. IEEE.

[3] Y Zha. INFORMATION UNCERTAINTY PRINCIPLE , 1989 .

[4] Esther Levin,et al. A statistical approach to learning and generalization in layered neural networks , 1989, COLT '89.

[5] Shun-ichi Amari,et al. Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[6] Panos J. Antsaklis,et al. The dependence identification neural network construction algorithm , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).