Activity Invariant Sets and Exponentially Stable Attractors of Linear Threshold Discrete-Time Recurrent Neural Networks

This technical note proposes to study the activity invariant sets and exponentially stable attractors of linear threshold discrete-time recurrent neural networks. The concept of activity invariant sets deeply describes the property of an invariant set by that the activity of some neurons keeps invariant all the time. Conditions are obtained for locating activity invariant sets. Under some conditions, it shows that an activity invariant set can have one equilibrium point which attracts exponentially all trajectories starting in the set. Since the attractors are located in activity invariant sets, each attractor has binary pattern and also carries analog information. Such results can provide new perspective to apply attractor networks for applications such as group winner-take-all, associative memory, etc.

[1]  Zhang Yi,et al.  Convergence Analysis of Recurrent Neural Networks , 2003, Network Theory and Applications.

[2]  Xiaohui Xie,et al.  Selectively Grouping Neurons in Recurrent Networks of Lateral Inhibition , 2002, Neural Computation.

[3]  Kay Chen Tan,et al.  Neural Networks: Computational Models and Applications , 2007 .

[4]  Kay Chen Tan,et al.  Qualitative analysis for recurrent neural networks with linear threshold transfer functions , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  H. Sompolinsky,et al.  Theory of orientation tuning in visual cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Chih-Chiang Cheng,et al.  A decentralized model reference adaptive variable structure controller for large-scale time-varying delay systems , 2003, IEEE Trans. Autom. Control..

[7]  Zhang Yi,et al.  Multistability Analysis for Recurrent Neural Networks with Unsaturating Piecewise Linear Transfer Functions , 2003, Neural Computation.

[8]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[9]  Anthony J. Calise,et al.  Coordinated decentralized adaptive output feedback control of interconnected systems , 2004, Proceedings of the 2004 American Control Conference.

[10]  Randal W. Beard,et al.  Consensus seeking in multiagent systems under dynamically changing interaction topologies , 2005, IEEE Transactions on Automatic Control.

[11]  Kay Chen Tan,et al.  Dynamics analysis and analog associative memory of networks with LT neurons , 2006, IEEE Transactions on Neural Networks.

[12]  C. Koch,et al.  Recurrent excitation in neocortical circuits , 1995, Science.

[13]  R. Olfati-Saber,et al.  Swarms on Sphere: A Programmable Swarm with Synchronous Behaviors like Oscillator Networks , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[14]  H. Sebastian Seung,et al.  Permitted and Forbidden Sets in Symmetric Threshold-Linear Networks , 2003, Neural Computation.

[15]  Richard H. R. Hahnloser,et al.  On the piecewise analysis of networks of linear threshold neurons , 1998, Neural Networks.

[16]  Zhang Yi,et al.  Multiperiodicity and Attractivity of Delayed Recurrent Neural Networks With Unsaturating Piecewise Linear Transfer Functions , 2008, IEEE Transactions on Neural Networks.

[17]  K. Tan,et al.  Dynamic stability conditions for Lotka-Volterra recurrent neural networks with delays. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Misha Tsodyks,et al.  From , 2020, Definitions.

[19]  Heiko Wersing,et al.  A Competitive-Layer Model for Feature Binding and Sensory Segmentation , 2001, Neural Computation.

[20]  Richard Hans Robert Hahnloser,et al.  Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit , 2000, Nature.

[21]  Heiko Wersing,et al.  Dynamical Stability Conditions for Recurrent Neural Networks with Unsaturating Piecewise Linear Transfer Functions , 2001, Neural Computation.

[22]  T. Carroll,et al.  MASTER STABILITY FUNCTIONS FOR SYNCHRONIZED COUPLED SYSTEMS , 1999 .

[23]  Zhang Yi,et al.  Recurrent Neural Networks with Unsaturating Piecewise Linear Activation Functions , 2004 .

[24]  Sue Ann Campbell,et al.  Frustration, Stability, and Delay-Induced Oscillations in a Neural Network Model , 1996, SIAM J. Appl. Math..