Backpropagation for linearly-separable patterns: A detailed analysis

A sufficient condition for learning without local minima in multilayered networks is proposed. A fundamental assumption on the network architecture is removed. It is proved that the conclusions drawn by M. Gori and A. Tesi (IEEE Trans. Pattern Anal. Mach. Intell., vol.14, no.1, pp.76-86, (1992)) also hold provided that the weight matrix associated with the hidden and output layer is pyramidal and has full rank. The analysis is carried out by using least mean squares (LMS)-threshold cost functions, which allow the identification of spurious and structural local minima.<<ETX>>

[1]  J. Slawny,et al.  Back propagation fails to separate where perceptrons succeed , 1989 .

[2]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[3]  YoungJu Choie,et al.  Local minima and back propagation , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[4]  Yih-Fang Huang,et al.  Bounds on the number of hidden neurons in multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[5]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[6]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[7]  Alberto Tesi,et al.  On the Problem of Local Minima in Backpropagation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  D. R. Hush,et al.  Error surfaces for multi-layer perceptrons , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.