The thrust of this paper is to present a new class of dynamical systems for neural computation. Our approach is motivated by an attempt to remove one of the most fundamental limitations of artificial neural networks — their rigid behavior compared with even the simplest biological systems. It exploits the concept of terminal attractors and repellers. We demonstrate that non-Lipschitzian dynamics, based upon the failure of the Lipschitz condition at repellers, exhibits a new qualitative effect — a multi-choice response to periodic external excitations. This property enables the construction of a substantially new class of dynamical systems — the unpredictable systems, represented in the form of coupled activation and learning dynamical equations, which exhibit two pathological characteristics. Firstly, such systems have zero Jacobian. As a result of that, they have an infinite number of equilibrium points which occupy curves, surfaces or hypersurfaces. Secondly, at all these equilibrium points the Lipschitz condition fails, so the equilibrium points become terminal attractors or repellers depending upon the sign of the periodic excitation. Both of these pathological characteristics result in multi-choice responses and lead to unpredictability. We show that the unpredictable systems can be controlled by sign strings, which uniquely define the systems behavior by specifying the direction of the motions at the critical points. By changing the combinations of signs in the code strings a system can reproduce any prescribed behavior to a prescribed accuracy. That is why such unpredictable systems driven by sign strings are extremely flexible and can be exploited as a powerful tool for complex pattern recognition.
[1]
Michail Zak,et al.
Terminal attractors in neural networks
,
1989,
Neural Networks.
[2]
E M Harth,et al.
Brain functions and neural dynamics.
,
1970,
Journal of theoretical biology.
[3]
Jacob Barhen,et al.
Neural networks with creative dynamics
,
1990
.
[4]
S. Daan,et al.
Two coupled oscillators: simulations of the circadian pacemaker in mammalian activity rhythms.
,
1978,
Journal of theoretical biology.
[5]
E M Harth,et al.
Dynamics of neural structures.
,
1970,
Journal of theoretical biology.
[6]
M. Zak.
Non-Lipschitzian dynamics for neural net modelling
,
1989
.
[7]
J J Hopfield,et al.
Neurons with graded response have collective computational properties like those of two-state neurons.
,
1984,
Proceedings of the National Academy of Sciences of the United States of America.
[8]
Sandeep Gulati,et al.
Neutral learning of constrained nonlinear transformations
,
1989,
Computer.