Chapter 17 Geometry of neural networks: Natural gradient for learning

[1]  A. Wald.  Note on the Consistency of the Maximum Likelihood Estimate , 1949 .

[2]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[3]  Theodor Bröcker,et al.  Introduction to Differential Topology , 1982 .

[4]  Shun-ichi Amari,et al.  Differential geometry of statistical inference , 1983 .

[5]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[6]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[7]  J. Rissanen.  Stochastic complexity and the mdl principle , 1987 .

[8]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[9]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[10]  M. Murray,et al.  Differential Geometry and Statistics , 1993 .

[11]  Robert Hecht-Nielsen,et al.  On the Geometry of Feedforward Neural Network Error Surfaces , 1993, Neural Computation.

[12]  Katsuyuki Hagiwara,et al.  On the problem of applying AIC to determine the structure of a layered feedforward neural network , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[13]  Saad,et al.  On-line learning in soft committee machines. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[14]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[15]  Kenji Fukumizu,et al.  A Regularity Condition of the Information Matrix of a Multilayer Perceptron Network , 1996, Neural Networks.

[16]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[17]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[18]  David Saad,et al.  On-Line Learning in Neural Networks , 1999 .

[19]  M. Rattray,et al.  Analysis of natural gradient descent for multilayer neural networks , 1999, cond-mat/9901212.

[20]  M. Rattray,et al.  MATRIX MOMENTUM FOR PRACTICAL NATURAL GRADIENT LEARNING , 1999 .

[21]  Kenji Fukumizu,et al.  Adaptive Method of Realizing Natural Gradient Learning for Multilayer Perceptrons , 2000, Neural Computation.

[22]  Kenji Fukumizu,et al.  Local minima and plateaus in hierarchical structures of multilayer perceptrons , 2000, Neural Networks.

[23]  Kenji Fukumizu,et al.  Adaptive natural gradient learning algorithms for various stochastic models , 2000, Neural Networks.