Near-Optimal Design of Safe Output Feedback Controllers from Noisy Data

Recent work in data-driven control has revived behavioral theory to perform a variety of complex control tasks, by directly plugging libraries of past inputoutput trajectories into optimal control problems. Despite recent advances, two key aspects remain unclear when the data are corrupted by noise: how can safety be guaranteed, and to what extent is the control performance affected? In this work, we provide a quantitative answer to these questions. In particular, we formulate a robustly safe version of the recently introduced Behavioral InputOutput Parametrization (BIOP) for the optimal predictive control of unknown constrained systems. The proposed framework has three main advantages: 1) it allows one to safely operate the system while explicitly quantifying, as a function of the noise level corrupting the data, how much the performance degrades, 2) it can be used in combination with state-of-the-art impulse response estimators, and finally, being a data-driven approach, 3) the state-space parameters and the initial state need not be specified for controller synthesis. We corroborate our results through numerical experiments.

[1]  Mario Zanon,et al.  Safe Reinforcement Learning Using Robust MPC , 2019, IEEE Transactions on Automatic Control.

[2]  Nikolai Matni,et al.  Safely Learning to Control the Constrained Linear Quadratic Regulator , 2018, 2019 American Control Conference (ACC).

[3]  Martin J. Wainwright,et al.  Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems , 2018, AISTATS.

[4]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[5]  Nikolai Matni,et al.  Robust Closed-loop Model Predictive Control via System Level Synthesis , 2019, 2020 59th IEEE Conference on Decision and Control (CDC).

[6]  Nikolai Matni,et al.  Data-Driven System Level Synthesis , 2020, L4DC.

[7]  Fabio Pasqualetti,et al.  Data-driven control of complex networks , 2020, Nature Communications.

[8]  Benjamin Recht,et al.  Certainty Equivalent Control of LQR is Efficient , 2019, ArXiv.

[9]  Benjamin Recht,et al.  A Tour of Reinforcement Learning: The View from Continuous Control , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[10]  M. Kanat Camlibel,et al.  From Noisy Data to Feedback Controllers: Nonconservative Design via a Matrix S-Lemma , 2022, IEEE Transactions on Automatic Control.

[11]  Nikolai Matni,et al.  Efficient Learning of Distributed Linear-Quadratic Control Policies , 2020, SIAM J. Control. Optim..

[12]  Alexandre Proutiere,et al.  Poisoning Attacks against Data-Driven Control Methods , 2021, 2021 American Control Conference (ACC).

[13]  Nikolai Matni,et al.  On the Sample Complexity of the Linear Quadratic Regulator , 2017, Foundations of Computational Mathematics.

[14]  Bart De Moor,et al.  A note on persistency of excitation , 2005, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[15]  Babak Hassibi,et al.  Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems , 2020, NeurIPS.

[16]  Maryam Kamgarpour,et al.  An Input–Output Parametrization of Stabilizing Controllers: Amidst Youla and System Level Synthesis , 2019, IEEE Control Systems Letters.

[17]  Maryam Kamgarpour,et al.  Sample Complexity of Linear Quadratic Gaussian (LQG) Control for Output Feedback Systems , 2020, L4DC.

[18]  George J. Pappas,et al.  Sample Complexity of Kalman Filtering for Unknown Systems , 2019, L4DC.

[19]  Giancarlo Ferrari-Trecate,et al.  A Behavioral Input-Output Parametrization of Control Policies with Suboptimality Guarantees , 2021, 2021 60th IEEE Conference on Decision and Control (CDC).

[20]  J. Kiefer,et al.  Sequential minimax search for a maximum , 1953 .

[21]  Pietro Tesi,et al.  Formulas for Data-Driven Control: Stabilization, Optimality, and Robustness , 2019, IEEE Transactions on Automatic Control.

[22]  Maryam Kamgarpour,et al.  Unified Approach to Convex Robust Distributed Control Given Arbitrary Information Structures , 2017, IEEE Transactions on Automatic Control.

[23]  Eric C. Kerrigan,et al.  Optimization over state feedback policies for robust control with constraints , 2006, Autom..

[24]  Mingzhou Yin,et al.  Maximum Likelihood Estimation in Data-Driven Modeling and Control , 2021, IEEE Transactions on Automatic Control.

[25]  John Lygeros,et al.  Data-Enabled Predictive Control: In the Shallows of the DeePC , 2018, 2019 18th European Control Conference (ECC).

[26]  Frank Allgöwer,et al.  Data-Driven Model Predictive Control With Stability and Robustness Guarantees , 2019, IEEE Transactions on Automatic Control.

[27]  Maryam Kamgarpour,et al.  Learning the Globally Optimal Distributed LQ Regulator , 2019, L4DC.

[28]  Pietro Tesi,et al.  Low-complexity Learning of Linear Quadratic Regulators from Noisy Data , 2020, Autom..

[29]  Tamer Basar,et al.  Policy Optimization for H2 Linear Control with H∞ Robustness Guarantee: Implicit Regularization and Global Convergence , 2020, L4DC.

[30]  Mingzhou Yin,et al.  Experiment design for impulse response identification with signal matrix models , 2021, IFAC-PapersOnLine.

[31]  Paolo Rapisarda,et al.  Data-driven simulation and control , 2008, Int. J. Control.

[32]  John Lygeros,et al.  An Extended Kalman Filter for Data-Enabled Predictive Control , 2020, IEEE Control Systems Letters.

[33]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[34]  Colin N. Jones,et al.  Nonlinear Data-Enabled Prediction and Control , 2021, L4DC.

[35]  Alberto Bemporad,et al.  Reducing conservativeness in predictive control of constrained systems with disturbances , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[36]  John Lygeros,et al.  Quadratic Regularization of Data-Enabled Predictive Control: Theory and Application to Power Converter Experiments , 2021, IFAC-PapersOnLine.

[37]  Sergio Lucia,et al.  On the relationship between data-enabled predictive control and subspace predictive control , 2021, 2021 European Control Conference (ECC).

[38]  Sham M. Kakade,et al.  Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator , 2018, ICML.

[39]  L.C.G.J.M. Habets,et al.  Book review: Introduction to mathematical systems theory, a behavioral approach , 2000 .

[40]  Anuradha M. Annaswamy,et al.  Systems & Control for the future of humanity, research agenda: Current and future roles, impact and grand challenges , 2017, Annu. Rev. Control..

[41]  Samet Oymak,et al.  Non-asymptotic Identification of LTI Systems from a Single Trajectory , 2018, 2019 American Control Conference (ACC).

[42]  John Lygeros,et al.  Distributionally Robust Chance Constrained Data-Enabled Predictive Control , 2020, IEEE Transactions on Automatic Control.

[43]  Stephen P. Boyd,et al.  Disciplined quasiconvex programming , 2019, Optimization Letters.

[44]  Ivan Markovsky,et al.  Bridging Direct and Indirect Data-Driven Control Formulations via Regularizations and Relaxations , 2021, IEEE Transactions on Automatic Control.