$l_2$ Induced Norm Analysis of Discrete-Time LTI Systems for Nonnegative Input Signals and Its Application to Stability Analysis of Recurrent Neural Networks

In this paper, we focus on the "positive" $l_2$ induced norm of discrete-time linear time-invariant systems where the input signals are restricted to be nonnegative. To cope with the nonnegativity of the input signals, we employ copositive programming as the mathematical tool for the analysis. Then, by applying an inner approximation to the copositive cone, we derive numerically tractable semidefinite programming problems for the upper and lower bound computation of the "positive" $l_2$ induced norm. This norm is typically useful for the stability analysis of feedback systems constructed from an LTI system and nonlinearities where the nonlinear elements provide only nonnegative signals. As a concrete example, we illustrate the usefulness of the "positive" $l_2$ induced norm for the stability analysis of recurrent neural networks with activation functions being rectified linear units.

[1]  Victor Magron,et al.  A hierarchy of spectral relaxations for polynomial optimization , 2020, Mathematical Programming Computation.

[2]  Teruki Kato,et al.  Analysis of Positive Systems Using Copositive Programming , 2020, IEEE Control Systems Letters.

[3]  Shahrokh Valaee,et al.  Recent Advances in Recurrent Neural Networks , 2017, ArXiv.

[4]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[5]  Carsten W. Scherer,et al.  Full‐block multipliers for repeated, slope‐restricted scalar nonlinearities , 2017 .

[6]  C. Scherer,et al.  Absolute stability analysis of discrete time feedback interconnections , 2017 .

[7]  Dimitri Peaucelle,et al.  Analysis and Synthesis of Interconnected Positive Systems , 2017, IEEE Transactions on Automatic Control.

[8]  Anders Rantzer,et al.  On the Kalman-Yakubovich-Popov Lemma for Positive Systems , 2012, IEEE Transactions on Automatic Control.

[9]  Huaguang Zhang,et al.  A Comprehensive Review of Stability Analysis of Continuous-Time Recurrent Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Jean B. Lasserre,et al.  New approximations for the cone of copositive matrices and its dual , 2010, Mathematical Programming.

[11]  P. Olver.  Nonlinear Systems , 2013 .

[12]  Sophie Tarbouriech,et al.  Stability and Stabilization of Linear Systems with Saturating Actuators , 2011 .

[13]  Mirjam Dür,et al.  Copositive Programming – a Survey , 2010 .

[14]  Peter J. C. Dickinson.  AN IMPROVED CHARACTERISATION OF THE INTERIOR OF THE COMPLETELY POSITIVE CONE , 2010 .

[15]  Mirjam Dür,et al.  Interior points of the completely positive cone. , 2008 .

[16]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[17]  Carsten W. Scherer,et al.  LMI Relaxations in Robust Control , 2006, Eur. J. Control.

[18]  J. Geromel,et al.  Extended H 2 and H norm characterizations and controller parametrizations for discrete-time systems , 2002 .

[19]  Etienne de Klerk,et al.  Aspects of Semidefinite Programming , 2002 .

[20]  S. Rinaldi,et al.  Positive Linear Systems: Theory and Applications , 2000 .

[21]  A. Rantzer.  On the Kalman-Yakubovich-Popov lemma , 1996 .

[22]  S. Bittanti,et al.  Analysis of discrete-time linear periodic systems , 1996 .

[23]  A. Rantzer,et al.  System analysis via integral quadratic constraints , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[24]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[25]  安藤 毅.  Completely positive matrices , 1991 .

[26]  P. Falb,et al.  Stability Conditions for Systems with Monotone and Slope-Restricted Nonlinearities , 1968 .