Dynamic Analysis of a General Class of Winner-Take-All Competitive Neural Networks

This paper studies a general class of dynamical neural networks with lateral inhibition, exhibiting winner-take-all (WTA) behavior. These networks are motivated by a metal-oxide-semiconductor field effect transistor (MOSFET) implementation of neural networks, in which mutual competition plays a very important role. We show that for a fairly general class of competitive neural networks, WTA behavior exists. Sufficient conditions for the network to have a WTA equilibrium are obtained, and rigorous convergence analysis is carried out. The conditions for the network to have the WTA behavior obtained in this paper provide design guidelines for the network implementation and fabrication. We also demonstrate that whenever the network gets into the WTA region, it will stay in that region and settle down exponentially fast to the WTA point. This provides a speeding procedure for the decision making: as soon as it gets into the region, the winner can be declared. Finally, we show that this WTA neural network has a self-resetting property, and a resetting principle is proposed.

[1]  Jar-Ferr Yang,et al.  A general mean-based iterative winner-take-all neural network , 1995, IEEE Trans. Neural Networks.

[2]  Bard Ermentrout,et al.  Complex dynamics in winner-take-all neural nets with slow inhibition , 1992, Neural Networks.

[3]  Yaser S. Abu-Mostafa,et al.  On the K-Winners-Take-All Network , 1988, NIPS.

[4]  Bahram Nabet,et al.  Sensory neural networks - lateral inhibiton , 1991 .

[5]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[6]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[7]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[8]  S. Grossberg.  Competition, Decision, and Consensus , 1978 .

[9]  A. Michel,et al.  Stability analysis of differential inclusions in Banach space with applications to nonlinear systems with time delays , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[10]  H. Ye,et al.  Robust stability of nonlinear time-delay systems with applications to neural networks , 1996 .

[11]  Anthony N. Michel,et al.  Robustness and perturbation analysis of a class of artificial neural networks , 1994, Neural Networks.

[12]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[13]  Teuvo Kohonen,et al.  Physiological interpretation of the Self-Organizing Map algorithm , 1993, Neural Networks.

[14]  Shyang Chang,et al.  A new winners-take-all architecture in artificial neural networks , 1994, IEEE Trans. Neural Networks.

[15]  M. Hirsch.  Systems of di erential equations which are competitive or cooperative I: limit sets , 1982 .

[16]  John Lazzaro,et al.  Winner-Take-All Networks of O(N) Complexity , 1988, NIPS.

[17]  Bahram Nabet,et al.  Stability of a class of directionally sensitive asymmetric nonlinear neural networks , 1996, Neural Networks.

[18]  Morris W. Hirsch,et al.  Convergent activation dynamics in continuous time networks , 1989, Neural Networks.

[19]  Teuvo Kohonen,et al.  Physiological interpretationm of the self-organizing map algorithm , 1993 .

[20]  B. V. K. Vijaya Kumar,et al.  Emulating the dynamics for a class of laterally inhibited neural networks , 1989, Neural Networks.

[21]  Samuel Kaski,et al.  Winner-take-all networks for physiological models of competitive learning , 1994, Neural Networks.

[22]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[23]  Jonathan S. Kane,et al.  Optoelectronic winner-take-all VLSI shunting neural network , 1995, IEEE Trans. Neural Networks.

[24]  Corneliu A. Marinov,et al.  Another K-winners-take-all analog neural network , 2000, IEEE Trans. Neural Networks Learn. Syst..

[25]  C. A. Desoer,et al.  Nonlinear Systems Analysis , 1978 .

[27]  Krzysztof Wawryn,et al.  Current Mode Circuits for Programmable WTA Neural Network , 2001 .

[28]  M. Vidyasagar,et al.  Nonlinear systems analysis (2nd ed.) , 1993 .

[29]  V. Lakshmikantham,et al.  Differential and integral inequalities : theory and applications , 1969 .

[30]  S. Grossberg.  Contour Enhancement , Short Term Memory , and Constancies in Reverberating Neural Networks , 1973 .

[31]  S. Arik,et al.  Stability analysis of dynamical neural networks , 1997 .