A classification-based approach to the optimal control of affine switched systems

This paper deals with the optimal control of discrete-time switched systems, characterized by a finite set of operating modes, each one associated with given affine dynamics. The objective is the design of the switching law so as to minimize an infinite-horizon expected cost, that penalizes frequent switchings. The optimal switching law is computed off-line, which allows an efficient online operation of the control via a state feedback policy. The latter associates a mode to each state and, as such, can be viewed as a classifier. In order to train such classifier-type controller one needs first to generate a set of training data in the form of optimal state-mode pairs. In the considered setting, this involves solving a Mixed Integer Quadratic Programming (MIQP) problem for each pair. A key feature of the proposed approach is the use of a classification method that provides guarantees on the generalization properties of the classifier. The approach is tested on a multi-room heating control problem.

[1]  Marco C. Campi,et al.  Classification with guaranteed probability of error , 2010, Machine Learning.

[2]  Franco Blanchini,et al.  Discrete‐time control for switched positive systems with application to mitigating viral escape , 2011 .

[3]  John Langford,et al.  Relating reinforcement learning performance to classification performance , 2005, ICML '05.

[4]  Alberto Bemporad,et al.  Piecewise linear optimal controllers for hybrid systems , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[5]  V. Borkar,et al.  A unified framework for hybrid control: model and optimal control theory , 1998, IEEE Trans. Autom. Control..

[6]  Alessandro Lazaric,et al.  Analysis of a Classification-based Policy Iteration Algorithm , 2010, ICML.

[7]  M. Egerstedt,et al.  Optimization of switched-mode systems with switching costs , 2008, 2008 American Control Conference.

[8]  A. Giua,et al.  Optimal control of switched autonomous linear systems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[9]  Alberto Bemporad,et al.  Control of systems integrating logic, dynamics, and constraints , 1999, Autom..

[10]  Alessandro Giua,et al.  STABILIZATION OF SWITCHED SYSTEMS VIA OPTIMAL CONTROL , 2005 .

[11]  Sven Leyffer,et al.  Numerical Experience with Lower Bounds for MIQP Branch-And-Bound , 1998, SIAM J. Optim..

[12]  Patrizio Colaneri,et al.  Switching Strategies to Mitigate HIV Mutation , 2014, IEEE Transactions on Control Systems Technology.

[13]  Alberto Bemporad,et al.  Optimal control of continuous-time switched affine systems , 2006, IEEE Transactions on Automatic Control.

[14]  Ansgar Fehnker,et al.  Benchmarks for Hybrid Systems Verification , 2004, HSCC.

[15]  R. Tempo,et al.  On the sample complexity of randomized approaches to the analysis and design under uncertainty , 2010, Proceedings of the 2010 American Control Conference.

[16]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[17]  Ioannis Rexakis,et al.  Directed exploration of policy space using support vector classifiers , 2011, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[18]  Michail G. Lagoudakis,et al.  Reinforcement Learning as Classification: Leveraging Modern Classifiers , 2003, ICML.

[19]  A. Rantzer,et al.  Piecewise linear quadratic optimal control , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[20]  Alberto Bemporad,et al.  Dynamic programming for constrained optimal control of discrete-time linear hybrid systems , 2005, Autom..

[21]  Franco Blanchini,et al.  Continuous-time optimal control for switched positive systems with application to mitigating viral escape , 2010 .