On the stability of neural networks with arbitrary weights

Dynamical properties of a general neural network are discussed. A condition for the neural activation dynamics being stable is derived. The stability condition does not put any symmetry restriction on the weights. The transitions from stable to non-stable dynamics are analysed and their analogy to phase transitions in statistical mechanics discussed. In general, the network dynamics can violate the stability condition. To avoid that happening, a method is introduced which makes the dynamics adaptive such that the stability condition is sustained. Relations to some previous works on stability are discussed.

[1]  Alan S. Lapedes,et al.  A self-optimizing, nonsymmetrical neural net for content addressable memory and pattern recognition , 1986 .

[2]  Sverrir Olafsson,et al.  SOME DYNAMICAL PROPERTIES OF NEURAL NETWORKS , 1992 .

[3]  D. Jordan,et al.  Nonlinear Ordinary Differential Equations: An Introduction for Scientists and Engineers , 1979 .

[4]  Morris W. Hirsch,et al.  Convergent activation dynamics in continuous time networks , 1989, Neural Networks.

[5]  J. Hopfield.  Neurons withgraded response havecollective computational properties likethoseoftwo-state neurons , 1984 .

[6]  Kiyotoshi Matsuoka,et al.  Stability conditions for nonlinear continuous neural networks with asymmetric connection weights , 1992, Neural Networks.

[7]  Luis B. Almeida.  Back propagation in non-feedforward networks , 1989 .

[9]  A. B. Farnell.  Limits for the characteristic roots of a matrix , 1944 .

[10]  B. Noble.  Applied Linear Algebra , 1969 .

[11]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[12]  A.N. Michel,et al.  Associative memories via artificial neural networks , 1990, IEEE Control Systems Magazine.

[13]  Fernando J. Pineda,et al.  Generalization of Back propagation to Recurrent and Higher Order Neural Networks , 1987, NIPS.

[14]  A. Brauer.  Limits for the characteristic roots of a matrix. III , 1946 .

[15]  Allon Guez,et al.  On the stability, storage capacity, and design of nonlinear continuous neural networks , 1988, IEEE Trans. Syst. Man Cybern..

[16]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .