Long-Short Term Memory Neural Network Stability and Stabilization using Linear Matrix Inequalities

A global asymptotic stability condition for Long Short-Term Memory neural networks is presented in this paper. A linear matrix inequality optimization problem is used to describe this global stability condition. The linear matrix inequality formulation can be viewed as a way for stabilization of Long Short-Term Memory neural networks since the networks' weight matrices and biases can be essentially treated as control variables. The condition and how to compute numerical values for the weight matrices and biases are illustrated by some examples.

[1]  Daniela Fischer.  Differential Equations Dynamical Systems And An Introduction To Chaos , 2016 .

[2]  J. P. Lasalle.  The stability and control of discrete processes , 1986 .

[3]  Boris Murmann,et al.  Some Local Stability Properties of an Autonomous Long Short-Term Memory Neural Network Model , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[4]  D. Siljak,et al.  Robust stability and stabilization of discrete-time non-linear systems: The LMI approach , 2001 .

[5]  Nils Bertschinger,et al.  Real-Time Computation at the Edge of Chaos in Recurrent Neural Networks , 2004, Neural Computation.

[6]  P. Holmes,et al.  Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields , 1983, Applied Mathematical Sciences.

[7]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[9]  Thomas Laurent,et al.  A recurrent neural network without chaos , 2016, ICLR.

[10]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[11]  D. Ho,et al.  Robust stabilization for a class of discrete-time non-linear systems via output feedback: The unified LMI approach , 2003 .

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.