Data-driven distributionally robust control of partially observable jump linear systems

We study safe, data-driven control of (Markov) jump linear systems with unknown transition probabilities, where both the discrete mode and the continuous state are to be inferred from output measurements. To this end, we develop a receding horizon estimator which uniquely identifies a sub-sequence of past mode transitions and the corresponding continuous state, allowing for arbitrary switching behavior. Unlike traditional approaches to mode estimation, we do not require an offline exhaustive search over mode sequences to determine the size of the observation window, but rather select it online. If the system is weakly mode observable, the window size will be upper bounded, leading to a finite-memory observer. We integrate the estimation procedure with a simple distributionally robust controller, which hedges against misestimations of the transition probabilities due to finite sample sizes. As additional mode transitions are observed, the used ambiguity sets are updated, resulting in continual improvements of the control performance. The practical applicability of the approach is illustrated on small numerical examples.

[1]  Panagiotis Patrinos,et al.  Learning-Based Distributionally Robust Model Predictive Control of Markovian Switching Systems with Guaranteed Stability and Recursive Feasibility , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[2]  Giorgio Battistelli,et al.  Active mode observability of switching linear systems , 2007, Autom..

[3]  Didier Maquin,et al.  Switching Time Estimation of Piecewise Linear Systems. Application to Diagnosis , 2003 .

[4]  Uwe D. Hanebeck,et al.  Optimal sequence-based LQG control over TCP-like networks subject to random transmission delays and packet losses , 2013, 2013 American Control Conference.

[5]  Panagiotis Patrinos,et al.  Learning-Based Risk-Averse Model Predictive Control for Adaptive Cruise Control with Stochastic Driver Models , 2020, ArXiv.

[6]  M. Kothare,et al.  Robust constrained model predictive control using linear matrix inequalities , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[7]  Panagiotis Patrinos,et al.  Data-driven distributionally robust LQR with multiplicative noise , 2019, L4DC.

[8]  Marco Pavone,et al.  A framework for time-consistent, risk-averse model predictive control: Theory and algorithms , 2014, 2014 American Control Conference.

[9]  Anders Rantzer,et al.  Observer Synthesis for Switched Discrete-Time Linear Systems using Relaxed Dynamic Programming , 2006 .

[10]  Amir Averbuch,et al.  Interacting Multiple Model Methods in Target Tracking: A Survey , 1988 .

[11]  Giorgio Battistelli,et al.  Mode-observability degree in discrete-time switching linear systems , 2014, Syst. Control. Lett..

[12]  Shie Mannor,et al.  Distributionally Robust Markov Decision Processes , 2010, Math. Oper. Res..

[13]  Magnus Egerstedt,et al.  ASYMPTOTIC OBSERVERS FOR DISCRETE-TIME SWITCHED LINEAR SYSTEMS , 2005 .

[14]  Giorgio Battistelli,et al.  Luenberger Observers For Switching Discrete-Time Linear Systems , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[15]  Stéphane Lecoeuche,et al.  A new state observer for switched linear systems using a non-smooth optimization approach , 2011, IEEE Conference on Decision and Control and European Control Conference.

[16]  M. Egerstedt,et al.  Pathwise observability and controllability are decidable , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[17]  Ehsan Elhamifar,et al.  Rank tests for the observability of discrete-time jump linear systems with inputs , 2009, 2009 American Control Conference.

[18]  Giorgio Battistelli,et al.  Active mode observation of switching systems based on set‐valued estimation of the continuous state , 2009 .

[19]  Alberto Bemporad,et al.  Stabilizing Model Predictive Control of Stochastic Constrained Linear Systems , 2012, IEEE Transactions on Automatic Control.

[20]  K. Ito,et al.  On State Estimation in Switching Environments , 1970 .

[21]  Sanjay Mehrotra,et al.  Distributionally Robust Optimization: A Review , 2019, ArXiv.

[22]  J. Norris.  Appendix: probability and measure , 1997 .

[23]  Pantelis Sopasakis,et al.  Safe Learning-Based Control of Stochastic Jump Linear Systems: a Distributionally Robust Approach , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[24]  Jamal Daafouz,et al.  Model-Based Modes Detection and Discernibility for Switched Affine Discrete-Time Systems , 2015, IEEE Transactions on Automatic Control.

[25]  Giorgio Battistelli,et al.  Receding-horizon estimation for switching discrete-time linear systems , 2005, IEEE Transactions on Automatic Control.

[26]  H. Chizeck,et al.  Controllability, observability and discrete-time markovian jump linear quadratic control , 1988 .

[27]  Magnus Egerstedt,et al.  Observability of Switched Linear Systems , 2004, HSCC.

[28]  Alberto Bemporad,et al.  Stochastic model predictive control for constrained discrete-time Markovian switching systems , 2014, Autom..

[29]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[30]  James Lam,et al.  Static output-feedback stabilization of discrete-time Markovian jump linear systems: A system augmentation approach , 2010, Autom..

[31]  WILLIAM P. BLAIR,et al.  Continuous-Time Regulation of a Class of Econometric Models , 1975, IEEE Transactions on Systems, Man, and Cybernetics.

[32]  Jitendra K. Tugnait,et al.  A detection-estimation scheme for state estimation in switching environments , 1979, Autom..

[33]  Aryeh Kontorovich,et al.  Minimax Learning of Ergodic Markov Chains , 2018, ALT.