Lyapunov-Net: A Deep Neural Network Architecture for Lyapunov Function Approximation

We develop a versatile deep neural network architecture, called Lyapunov-Net, to approximate Lyapunov functions of dynamical systems in high dimensions. Lyapunov-Net guarantees positive definiteness, and thus it can be easily trained to satisfy the negative orbital derivative condition, which only renders a single term in the empirical risk function in practice. This significantly reduces the number of hyper-parameters compared to existing methods. We also provide theoretical justifications on the approximation power of Lyapunov-Net and its complexity bounds. We demonstrate the efficiency of the proposed method on nonlinear dynamical systems involving up to 30 dimensional state spaces, and show that the proposed approach significantly outperforms the state-of-the-art methods.

[1]  Andreas Krause,et al.  The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems , 2018, CoRL.

[2]  M. M. Bayoumi,et al.  Feedback stabilization: control Lyapunov functions modelled by neural networks , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[3]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[4]  Aude Billard,et al.  Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions , 2014, Robotics Auton. Syst..

[5]  Sicun Gao,et al.  Neural Lyapunov Control , 2020, NeurIPS.

[6]  Stavros Petridis,et al.  Construction of Neural Network Based Lyapunov Functions , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[7]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[8]  Qiang Du,et al.  New Error Bounds for Deep ReLU Networks Using Sparse Grids , 2017, SIAM J. Math. Data Sci..

[9]  E Weinan,et al.  Exponential convergence of the deep neural network approximation for analytic functions , 2018, Science China Mathematics.

[10]  Philipp Petersen,et al.  Optimal approximation of piecewise smooth functions using deep ReLU neural networks , 2017, Neural Networks.

[11]  E Weinan,et al.  Overcoming the curse of dimensionality: Solving high-dimensional partial differential equations using deep learning , 2017, ArXiv.

[12]  Lars Grune.  Computing Lyapunov functions using deep neural networks , 2021 .

[13]  Frank L. Lewis,et al.  Neural Network Control Of Robot Manipulators And Non-Linear Systems , 1998 .

[14]  Fumin Zhang,et al.  An algorithm for computing robust forward invariant sets of two dimensional nonlinear systems , 2020 .

[15]  Moise Blanchard,et al.  The Representation Power of Neural Networks: Breaking the Curse of Dimensionality , 2020, ArXiv.

[16]  Aleksandr Beknazaryan,et al.  Analytic function approximation by path norm regularized deep networks , 2021 .

[17]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[18]  G. Serpen,et al.  Empirical approximation for Lyapunov functions with artificial neural nets , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[19]  Gang Bao,et al.  Weak Adversarial Networks for High-dimensional Partial Differential Equations , 2019, J. Comput. Phys..

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Haizhao Yang,et al.  Neural Network Approximation: Three Hidden Layers Are Enough , 2020, Neural Networks.

[22]  Liang Chen,et al.  A note on the expressive power of deep rectified linear unit networks in high‐dimensional spaces , 2019, Mathematical Methods in the Applied Sciences.

[23]  R. Srikant,et al.  Why Deep Neural Networks for Function Approximation? , 2016, ICLR.

[24]  Eduardo D. Sontag,et al.  Feedback Stabilization Using Two-Hidden-Layer Nets , 1991, 1991 American Control Conference.

[25]  J. A. A. Opschoor,et al.  Exponential ReLU DNN Expression of Holomorphic Maps in High Dimension , 2021, Constructive Approximation.

[26]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[27]  Dmitry Yarotsky,et al.  Error bounds for approximations with deep ReLU networks , 2016, Neural Networks.