Multistability and Instability of Neural Networks With Discontinuous Nonmonotonic Piecewise Linear Activation Functions

In this paper, we discuss the coexistence and dynamical behaviors of multiple equilibrium points for recurrent neural networks with a class of discontinuous nonmonotonic piecewise linear activation functions. It is proved that under some conditions, such n-neuron neural networks can have at least 5n equilibrium points, 3n of which are locally stable and the others are unstable, based on the contraction mapping theorem and the theory of strict diagonal dominance matrix. The investigation shows that the neural networks with the discontinuous activation functions introduced in this paper can have both more total equilibrium points and more locally stable equilibrium points than the ones with continuous Mexican-hat-type activation function or discontinuous two-level activation functions. An illustrative example with computer simulations is presented to verify the theoretical analysis.

[1]  Eugenius Kaszkurewicz,et al.  Solving systems of linear equations via gradient systems with discontinuous righthand sides: application to LS-SVM , 2005, IEEE Transactions on Neural Networks.

[2]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[3]  Chih-Wen Shih,et al.  Multiple Almost Periodic Solutions in Nonautonomous Delayed Neural Networks , 2007, Neural Computation.

[4]  Eva Kaslik,et al.  Impulsive hybrid discrete-time Hopfield neural networks with delays and multistability analysis , 2011, Neural Networks.

[5]  Huaguang Zhang,et al.  Dynamical stability analysis of multiple equilibrium points in time-varying delayed recurrent neural networks with discontinuous activation functions , 2012, Neurocomputing.

[6]  Qiankun Song,et al.  Multistability in Networks With Self-Excitation and High-Order Synaptic Connectivity , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[7]  Jinde Cao,et al.  Multistability in bidirectional associative memory neural networks , 2008 .

[8]  Chih-Wen Shih,et al.  Complete Stability in Multistable Delayed Neural Networks , 2009, Neural Computation.

[9]  M. Forti,et al.  Generalized Lyapunov approach for convergence of neural networks with discontinuous or non-Lipschitz activations , 2006 .

[10]  Jinde Cao,et al.  Multistability of neural networks with discontinuous activation function , 2008 .

[11]  Jinde Cao,et al.  Multistability of competitive neural networks with time-varying and distributed delays , 2009 .

[12]  Jinde Cao,et al.  Multistability and instability of delayed competitive neural networks with nondecreasing piecewise linear activation functions , 2013, Neurocomputing.

[13]  Jinde Cao,et al.  Delay-dependent multistability in recurrent neural networks , 2010, Neural Networks.

[14]  Mauro Forti.  M-matrices and global convergence of discontinuous neural networks , 2007, Int. J. Circuit Theory Appl..

[15]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[16]  Zarita Zainuddin,et al.  Function approximation using artificial neural networks , 2007 .

[17]  Zhigang Zeng,et al.  Analysis and design of associative memories based on recurrent neural network with discontinuous activation functions , 2012, Neurocomputing.

[18]  Huaguang Zhang,et al.  Multistability and multiperiodicity of delayed bidirectional associative memory neural networks with discontinuous activation functions , 2012, Appl. Math. Comput..

[19]  CHIH-WEN SHIH,et al.  Multistability in Recurrent Neural Networks , 2006, SIAM J. Appl. Math..

[20]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[21]  Zhenkun Huang,et al.  Multistability and multiperiodicity of high-order competitive neural networks with a general class of activation functions , 2012, Neurocomputing.

[22]  Edwin K. P. Chong,et al.  An analysis of a class of neural networks for solving linear programming problems , 1999, IEEE Trans. Autom. Control..

[23]  Wolfgang Porod,et al.  Analysis and synthesis of a class of neural networks: variable structure systems with infinite grain , 1989 .

[24]  M. Moraud.  Wavelet Networks , 2018, Foundations of Wavelet Networks and Applications.

[25]  Jinde Cao,et al.  Multistability and multiperiodicity of delayed Cohen–Grossberg neural networks with a general class of activation functions , 2008 .

[26]  Zhang Yi,et al.  Multistability of discrete-time recurrent neural networks with unsaturating piecewise linear activation functions , 2004, IEEE Transactions on Neural Networks.

[27]  Chang-Yuan Cheng,et al.  Multistability and convergence in delayed neural networks , 2007 .

[28]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Zhang Yi,et al.  Activity Invariant Sets and Exponentially Stable Attractors of Linear Threshold Discrete-Time Recurrent Neural Networks , 2009, IEEE Transactions on Automatic Control.

[30]  Zhigang Zeng,et al.  Multiperiodicity of Discrete-Time Delayed Neural Networks Evoked by Periodic External Inputs , 2006, IEEE Transactions on Neural Networks.

[31]  Tianping Chen,et al.  Coexistence and local stability of multiple equilibria in neural networks with piecewise linear nondecreasing activation functions , 2010, Neural Networks.

[32]  Tianping Chen,et al.  Multistability and New Attraction Basins of Almost-Periodic Solutions of Delayed Neural Networks , 2009, IEEE Transactions on Neural Networks.

[33]  Zhigang Zeng,et al.  Multistability of Two Kinds of Recurrent Neural Networks With Activation Functions Symmetrical About the Origin on the Phase Plane , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Zhigang Zeng,et al.  Multiperiodicity and Exponential Attractivity Evoked by Periodic External Inputs in Delayed Cellular Neural Networks , 2006, Neural Computation.

[35]  Zarita Zainuddin,et al.  AN EFFECTIVE AND NOVEL WAVELET NEURAL NETWORK APPROACH IN CLASSIFYING TYPE 2 DIABETICS , 2012 .

[36]  Zhigang Zeng,et al.  Multistability of Recurrent Neural Networks With Time-varying Delays and the Piecewise Linear Activation Function , 2010, IEEE Transactions on Neural Networks.

[37]  Xinghua Wang,et al.  Multiperiodicity of Periodically Oscillated Discrete-Time Neural Networks With Transient Excitatory Self-Connections and Sigmoidal Nonlinearities , 2010, IEEE Transactions on Neural Networks.

[38]  Eva Kaslik,et al.  Multiple periodic solutions in impulsive hybrid neural networks with delays , 2011, Appl. Math. Comput..

[39]  Zhang Yi,et al.  Multiperiodicity and Attractivity of Delayed Recurrent Neural Networks With Unsaturating Piecewise Linear Transfer Functions , 2008, IEEE Transactions on Neural Networks.

[40]  Tianping Chen,et al.  On Attracting Basins of Multiple Equilibria of a Class of Cellular Neural Networks , 2011, IEEE Transactions on Neural Networks.

[41]  Jinde Cao,et al.  Multistability of Second-Order Competitive Neural Networks With Nondecreasing Saturated Activation Functions , 2011, IEEE Transactions on Neural Networks.

[42]  M. Forti,et al.  Global convergence of neural networks with discontinuous neuron activations , 2003 .

[43]  Tianping Chen,et al.  Multistability of Neural Networks With Mexican-Hat-Type Activation Functions , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[44]  Zhigang Zeng,et al.  Multistability of Neural Networks With Time-Varying Delays and Concave-Convex Characteristics , 2012, IEEE Transactions on Neural Networks and Learning Systems.