论文信息 - Transients and asymptotics of natural gradient learning

Transients and asymptotics of natural gradient learning

We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

Magnus Rattray | David Saad

[1] David Saad,et al. On-line learning with adaptive back-propagation in two-layer networks , 1997 .

[2] Saad,et al. Exact solution for on-line learning in multilayer neural networks. , 1995, Physical review letters.

[3] Magnus Rattray,et al. Natural gradient descent for on-line learning , 1998 .

[4] Shun-ichi Amari,et al. Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[5] Magnus Rattray,et al. Globally optimal parameters for on-line learning in multilayer neural networks , 1997 .

[6] Shun-ichi Amari,et al. Complexity Issues in Natural Gradient Descent Method for Training Multilayer Perceptrons , 1998, Neural Computation.

[7] David Barber,et al. Finite-size effects in on-line learning of multilayer neural networks , 1996 .

[8] George Cybenko,et al. Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..