Comments on local minima free conditions in multilayer perceptrons

In this letter we point out that multilayer neural networks (MLP's) with either sigmoidal units or radial basis functions can be given a canonical form with positive interunits weights, which does not restrict the well-known MLP universal computational capabilities. We give some results on the local minima of the error function using this canonical form. In particular, we prove that the local minima free conditions established in previous works can be relaxed significantly.

[1]  Sontag,et al.  Backpropagation separates when perceptrons do , 1989 .

[2]  Alberto Tesi,et al.  On the Problem of Local Minima in Backpropagation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Marco Gori,et al.  On the problem of local minima in recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[4]  Ida G. Sprinkhuizen-Kuyper,et al.  The Error Surface of the Simplest XOR Network Has Only Global Minima , 1996, Neural Computation.

[5]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[6]  Marco Gori,et al.  Optimal learning in artificial neural networks: A review of theoretical results , 1996, Neurocomputing.

[7]  Eduardo D. Sontag,et al.  Backpropagation Can Give Rise to Spurious Local Minima Even for Networks without Hidden Layers , 1989, Complex Syst..

[8]  X H Yu,et al.  On the local minima free condition of backpropagation learning , 1995, IEEE Trans. Neural Networks.

[9]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[10]  J. Slawny,et al.  Back propagation fails to separate where perceptrons succeed , 1989 .

[11]  Robert Hecht-Nielsen,et al.  On the Geometry of Feedforward Neural Network Error Surfaces , 1993, Neural Computation.

[12]  Leonard G. C. Hamey,et al.  XOR has no local minima: A case study in neural network error surface analysis , 1998, Neural Networks.

[13]  Paolo Frasconi,et al.  Learning without local minima in radial basis function networks , 1995, IEEE Trans. Neural Networks.