Limit Set Dichotomy and Multistability for a Class of Cooperative Neural Networks With Delays

Recent papers have pointed out the interest to study convergence in the presence of multiple equilibrium points (EPs) (multistability) for neural networks (NNs) with nonsymmetric cooperative (nonnegative) interconnections and neuron activations modeled by piecewise linear (PL) functions. One basic difficulty is that the semiflows generated by such NNs are monotone but, due to the horizontal segments in the PL functions, are not eventually strongly monotone (ESM). This notwithstanding, it has been shown that there are subclasses of irreducible interconnection matrices for which the semiflows, although they are not ESM, enjoy convergence properties similar to those of ESM semiflows. The results obtained so far concern the case of cooperative NNs without delays. The goal of this paper is to extend some of the existing results to the relevant case of NNs with delays. More specifically, this paper considers a class of NNs with PL neuron activations, concentrated delays, and a nonsymmetric cooperative interconnection matrix A and delay interconnection matrix Aτ. The main result is that when A+Aτ satisfies a full interconnection condition, then the generated semiflows, which are monotone but not ESM, satisfy a limit set dichotomy analogous to that valid for ESM semiflows. It follows that there is an open and dense set of initial conditions, in the state space of continuous functions on a compact interval, for which the solutions converge toward an EP. The result holds in the general case where the NNs possess multiple EPs, i.e., is a result on multistability, and is valid for any constant value of the delays.

[1]  Marco Gilli,et al.  Stability of cellular neural networks and delayed cellular neural networks with nonpositive templates and nonmonotonic output functions , 1994 .

[2]  Hal L. Smith.  Monotone semiflows generated by functional differential equations , 1987 .

[3]  Zhigang Zeng,et al.  Complete stability of cellular neural networks with time-varying delays , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[4]  R. Westervelt,et al.  Stability of analog neural networks with delay. , 1989, Physical review. A, General physics.

[5]  YANGJin-xiang,et al.  Stability for Cellular Neural Networks with Delay , 2005 .

[6]  Jinde Cao,et al.  Delay-dependent multistability in recurrent neural networks , 2010, Neural Networks.

[7]  Mauro Forti,et al.  A note on the dichotomy of limit sets for cooperative CNNs with delays , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[8]  Stephen Grossberg,et al.  Absolute stability of global pattern formation and parallel memory storage by competitive neural networks , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Zhigang Zeng,et al.  Global asymptotic stability and global exponential stability of delayed cellular neural networks , 2005, IEEE Transactions on Circuits and Systems II: Express Briefs.

[10]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[11]  Alberto Tesi,et al.  A New Method to Analyze Complete stability of PWL Cellular Neural Networks , 2001, Int. J. Bifurc. Chaos.

[12]  Lihong Huang,et al.  Dynamics of smooth essentially strongly order-preserving semiflows with application to delay differential equations , 2008 .

[13]  Jifa Jiang,et al.  On the ω-limit Set Dichotomy of Cooperating Kolmogorov Systems , 2003 .

[14]  A. Michel,et al.  Robustness and Perturbation Analysis of a Class of Nonlinear Systems with Applications to Neural Networks , 1993, 1993 American Control Conference.

[15]  Mauro Forti,et al.  Lyapunov Method and Convergence of the Full-Range Model of CNNs , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[16]  Mauro Forti,et al.  Limit Set Dichotomy and Convergence of Cooperative Piecewise Linear Neural Networks , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[17]  Leon O. Chua,et al.  Stability and dynamics of delay-type general and cellular neural networks , 1992 .

[18]  Zidong Wang,et al.  Monostability and multistability of genetic regulatory networks with different types of regulation functions , 2010 .

[19]  Pauline van den Driessche,et al.  Global Attractivity in Delayed Hopfield Neural Network Models , 1998, SIAM J. Appl. Math..

[20]  Patrick Thiran,et al.  Dynamics and self-organization of locally coupled neural networks , 1996 .

[21]  A. Tesi,et al.  New conditions for global stability of neural networks with application to linear and quadratic programming problems , 1995 .

[22]  Mauro Forti,et al.  Convergence of a Class of Cooperative Standard Cellular Neural Network Arrays , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[23]  P. Thiran,et al.  An approach to information propagation in 1-D cellular neural networks. II. Global propagation , 1998 .

[24]  H. H. Schaefer.  Banach Lattices and Positive Operators , 1975 .

[25]  Lihong Huang,et al.  Convergence and stability for essentially strongly order-preserving semiflows , 2006 .

[26]  Morris W. Hirsch,et al.  Convergent activation dynamics in continuous time networks , 1989, Neural Networks.

[27]  A. Tesi,et al.  Existence and characterization of limit cycles in nearly symmetric neural networks , 2002 .

[28]  Leon O. Chua,et al.  On the universe of stable cellular neural networks , 1992, Int. J. Circuit Theory Appl..

[29]  Chih-Wen Shih,et al.  Complete stability for a Class of Cellular Neural Networks , 2001, Int. J. Bifurc. Chaos.

[30]  David Angeli,et al.  Attractors in coherent systems of differential equations , 2007, 0710.3507.

[31]  Tianping Chen,et al.  Coexistence and local stability of multiple equilibria in neural networks with piecewise linear nondecreasing activation functions , 2010, Neural Networks.

[32]  Rolf Unbehauen,et al.  Canonical piecewise-linear networks , 1995, IEEE Trans. Neural Networks.

[33]  Xuemei Li,et al.  Invariance principle and complete stability for cellular neural networks , 2006, IEEE Trans. Circuits Syst. II Express Briefs.

[34]  Hiroshi Matano,et al.  Existence of nontrivial unstable sets for equilibriums of strongly order-preserving systems , 1984 .

[35]  Wolfgang Walter,et al.  On strongly monotone flows , 1997 .

[36]  Patrick Thiran,et al.  An approach to information propagation in cellular neural networks - Part I : Local diffusion , 1998 .

[37]  A LeonO.EtAl.Chu,et al.  Linear and nonlinear circuits , 2014 .

[38]  A. Michel,et al.  Analysis and synthesis of a class of neural networks: linear systems operating on a closed hypercube , 1989 .

[39]  Jinde Cao,et al.  Multistability and multiperiodicity of delayed Cohen–Grossberg neural networks with a general class of activation functions , 2008 .

[40]  Hal L. Smith,et al.  Strongly order preserving semiflows generated by functional differential equations , 1991 .

[41]  Zhang Yi,et al.  Multistability Analysis for Recurrent Neural Networks with Unsaturating Piecewise Linear Transfer Functions , 2003, Neural Computation.

[42]  Jianhong Wu,et al.  Introduction to Functional Differential Equations , 2013 .

[43]  Wei Xing Zheng,et al.  A New Method for Complete Stability Analysis of Cellular Neural Networks With Time Delay , 2010, IEEE Transactions on Neural Networks.

[44]  Norikazu Takahashi,et al.  A new sufficient condition for complete stability of cellular neural networks with delay , 2000 .

[45]  L. Chua,et al.  On the complete stability of nonsymmetric cellular neural networks , 1998 .

[46]  J. Lei.  Monotone Dynamical Systems , 2013 .

[47]  Chris Cosner,et al.  Book Review: Monotone dynamical systems: An introduction to the theory of competitive and cooperative systems , 1996 .

[48]  Jin Xu,et al.  An improved result for complete stability of delayed cellular neural networks , 2005, Autom..

[49]  Horst R. Thieme,et al.  Convergence for strongly order-preserving semiflows , 1991 .

[50]  Wang,et al.  Qualitative analysis of Cohen-Grossberg neural networks with multiple delays. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[51]  Michael Peter Kennedy,et al.  Unifying the Tank and Hopfield linear programming circuit and the canonical nonlinear programming circuit of Chua and Lin , 1987 .

[52]  Lin-Bao Yang,et al.  Cellular neural networks: theory , 1988 .

[53]  Leon O. Chua,et al.  Stability of a class of nonreciprocal cellular neural networks , 1990 .

[54]  M. Grazzini,et al.  The dichotomy of omega-limit sets fails for cooperative standard CNNs , 2010, 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010).

[55]  Horst R. Thieme,et al.  Quasi convergence and stability for strongly order-preserving semiflows , 1990 .

[56]  Tianping Chen,et al.  On Attracting Basins of Multiple Equilibria of a Class of Cellular Neural Networks , 2011, IEEE Transactions on Neural Networks.

[57]  Amir F. Atiya,et al.  How delays affect neural dynamics and learning , 1994, IEEE Trans. Neural Networks.

[58]  Chih-Wen Shih,et al.  Convergent dynamics for multistable delayed neural networks , 2008 .

[59]  Chang-Yuan Cheng,et al.  Multistability and convergence in delayed neural networks , 2007 .

[60]  Wolfgang Porod,et al.  Qualitative analysis and synthesis of a class of neural networks , 1988 .

[61]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[62]  Eduardo D. Sontag,et al.  A cooperative system which does not satisfy the limit set dichotomy , 2005 .

[63]  Zhigang Zeng,et al.  Multistability of Recurrent Neural Networks With Time-varying Delays and the Piecewise Linear Activation Function , 2010, IEEE Transactions on Neural Networks.