论文信息 - Geometrical Perspective on Learning Behavior

Geometrical Perspective on Learning Behavior

This paper constructs a geometrical perspective to justify the slow learning period and fast learning period during training. It plots the error surfaces and the solution spaces in the input space for a single neuron with two inputs. It records various training paths in this space using the back-propagation (BP) training algorithm [6]. It finds the relations between the learning curve and training path.

Cheng-Yuan Liou | Jau-Chi Huang | Yen-Ting Kuo

[1] 黄承國. The Third Asian Fuzzy Systems Symposium(AFSS'98)報告 , 1998 .

[2] Paul J. Werbos,et al. Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[3] Cheng-Yuan Liou,et al. Self-Relaxation for Multilayer Perceptron , 1998 .

[4] Thomas M. Cover,et al. Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[5] Raymond L. Watrous. Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization , 1988 .

[6] Robert A. Jacobs,et al. Increased rates of convergence through learning rate adaptation , 1987, Neural Networks.

[7] Geoffrey E. Hinton,et al. Learning representations by back-propagating errors , 1986, Nature.

[8] A. Michel,et al. Analysis and synthesis of a class of neural networks: linear systems operating on a closed hypercube , 1989 .

[9] Sang-Hoon Oh,et al. An analysis of premature saturation in back propagation learning , 1993, Neural Networks.

[10] W. C. Miller,et al. A new acceleration technique for the backpropagation algorithm , 1993, IEEE International Conference on Neural Networks.