Randomized Algorithms for Data-Driven Stabilization of Stochastic Linear Systems

Data-driven control strategies for dynamical systems with unknown parameters are popular in theory and applications. An essential problem is to prevent stochastic linear systems becoming destabilized, due to the uncertainty of the decision-maker about the dynamical parameter. Two randomized algorithms are proposed for this problem, but the performance is not sufficiently investigated. Further, the effect of key parameters of the algorithms such as the magnitude and the frequency of applying the randomizations is not currently available. This work studies the stabilization speed and the failure probability of data-driven procedures. We provide numerical analyses for the performance of two methods: stochastic feedback, and stochastic parameter. The presented results imply that as long as the number of statistically independent randomizations is not too small, fast stabilization is guaranteed.

[1]  Ambuj Tewari,et al.  On adaptive Linear-Quadratic regulators , 2020, Autom..

[2]  S. Bittanti,et al.  ADAPTIVE CONTROL OF LINEAR TIME INVARIANT SYSTEMS: THE "BET ON THE BEST" PRINCIPLE ∗ , 2006 .

[3]  Ambuj Tewari,et al.  Input Perturbations for Adaptive Regulation and Learning , 2018, ArXiv.

[4]  Alexander Rakhlin,et al.  How fast can linear dynamical systems be learned? , 2018, ArXiv.

[5]  Alexander Rakhlin,et al.  Near optimal finite time identification of arbitrary linear dynamical systems , 2018, ICML.

[6]  Ambuj Tewari,et al.  Optimism-Based Adaptive Regulation of Linear-Quadratic Systems , 2017, IEEE Transactions on Automatic Control.

[7]  George J. Pappas,et al.  An Information Matrix Approach for State Secrecy , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[8]  Csaba Szepesvári,et al.  Regret Bounds for the Adaptive Control of Linear Quadratic Systems , 2011, COLT.

[9]  B. Nielsen.  Singular vector autoregressions with deterministic terms: Strong consistency and lag order determination , 2008 .

[10]  Ambuj Tewari,et al.  On Optimality of Adaptive Linear-Quadratic Regulators , 2018, ArXiv.

[11]  James Lam,et al.  Stabilization of Discrete-Time Nonlinear Uncertain Systems by Feedback Based on LS Algorithm , 2013, SIAM J. Control. Optim..

[12]  Rieko Osu,et al.  The central nervous system stabilizes unstable dynamics by learning optimal impedance , 2001, Nature.

[13]  Ambuj Tewari,et al.  Finite Time Identification in Unstable Linear Systems , 2017, Autom..

[14]  Han-Fu Chen,et al.  The AAstrom-Wittenmark self-tuning regulator revisited and ELS-based adaptive trackers , 1991 .

[15]  Alessandro Lazaric,et al.  LQG for Portfolio Optimization , 2016, 1611.00997.

[16]  Ambuj Tewari,et al.  Finite Time Adaptive Stabilization of LQ Systems , 2018, ArXiv.

[17]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[18]  Lei Guo,et al.  Global Stability/Instability of LS-Based Discrete-Time Adaptive Nonlinear Control , 1996 .

[19]  Ruth F. Curtain,et al.  Linear-quadratic control: An introduction , 1997, Autom..

[20]  Nikolai Matni,et al.  On the Sample Complexity of the Linear Quadratic Regulator , 2017, Foundations of Computational Mathematics.

[21]  Alessandro Lazaric,et al.  Improved Regret Bounds for Thompson Sampling in Linear Quadratic Control Problems , 2018, ICML.

[22]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[23]  Adel Javanmard,et al.  Efficient Reinforcement Learning for High Dimensional Linear Quadratic Systems , 2012, NIPS.