论文信息 - Dynamics of Learning In Hierarchical Models – Singularity and Milnor Attractor

Dynamics of Learning In Hierarchical Models – Singularity and Milnor Attractor

We study the dynamics of learning in a hierarchical model such as multilayer perceptron. Such a model includes singularities, which affects its dynamics seriously. The Milnor type attractors appear, because of the singularity. We will show its trajectories explicitly, and present the topological nature of the singularities.

[1] Shun-ichi Amari,et al. Singularities Affect Dynamics of Learning in Neuromanifolds , 2006, Neural Comput..

[2] Shun-ichi Amari,et al. Dynamics of Learning Near Singularities in Layered Networks , 2008, Neural Computation.

[3] Shun-ichi Amari,et al. Dynamics of Learning in Multilayer Perceptrons Near Singularities , 2008, IEEE Transactions on Neural Networks.

[4] Shun-ichi Amari,et al. Dynamics of learning near singularities in radial basis function networks , 2008, Neural Networks.

[5] Kenji Fukumizu,et al. Local minima and plateaus in hierarchical structures of multilayer perceptrons , 2000, Neural Networks.

[6] Shun-ichi Amari,et al. Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[7] J. Milnor. On the concept of attractor , 1985 .