Near-optimal strategies for nonlinear networked control systems using optimistic planning

We consider the scenario where a controller communicates with a general nonlinear plant via a network, and must optimize a performance index. The problem is modeled in discrete time and the admissible control inputs are constrained to belong to a finite set. Exploiting a recent optimistic planning algorithm from the artificial intelligence field, we propose two control strategies that take into account communication constraints induced by the use of the network. Both resulting algorithms have guaranteed near-optimality. In the first strategy, input sequences are transmitted to the plant at a fixed period, and we show bounded computation. In the second strategy, the algorithm decides the next transmission instant according to the last state measurement (leading to a self-triggered policy), working within a fixed computation budget. For this case, we guarantee long transmission intervals. Examples and simulation experiments are provided throughout the paper to illustrate the results.

[1]  Wei Zhang,et al.  Scheduling and feedback co-design for networked control systems , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[2]  Xiaofeng Wang,et al.  Self-Triggered Feedback Control Systems With Finite-Gain ${\cal L}_{2}$ Stability , 2009, IEEE Transactions on Automatic Control.

[3]  Michael L. Littman,et al.  Sample-Based Planning for Continuous Action Markov Decision Processes , 2011, ICAPS.

[4]  Anton Cervin,et al.  Sporadic event-based control of first-order linear stochastic systems , 2008, Autom..

[5]  Bart De Schutter,et al.  Optimistic planning for sparsely stochastic systems , 2011, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[6]  Rémi Munos,et al.  Optimistic Planning of Deterministic Systems , 2008, EWRL.

[7]  Daniel E. Quevedo,et al.  Self-Triggered Model Predictive Control for Network Scheduling and Control , 2012 .

[8]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[9]  Anton Cervin,et al.  Scheduling of event-triggered controllers on a shared network , 2008, 2008 47th IEEE Conference on Decision and Control.

[10]  Frank Allgöwer,et al.  Analysis of Networked Event-Based Control with a Shared Communication Medium: Part I – Pure ALOHA , 2011 .

[11]  Paulo Tabuada,et al.  Dynamic programming formulation of periodic event-triggered control: Performance Guarantees and co-design , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[12]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[13]  Antonio Bicchi,et al.  Delay compensation in packet-switching networked controlled systems , 2008, 2008 47th IEEE Conference on Decision and Control.

[14]  Frank Allgöwer,et al.  On the optimal sending rate for Networked Control Systems with a shared communication medium , 2011, IEEE Conference on Decision and Control and European Control Conference.

[15]  Claudio De Persis,et al.  Robust Self-Triggered Coordination With Ternary Controllers , 2012, IEEE Transactions on Automatic Control.

[16]  W. P. M. H. Heemels,et al.  Analysis of event-driven controllers for linear systems , 2008, Int. J. Control.

[17]  Sandra Hirche,et al.  On LQG joint optimal scheduling and control under communication constraints , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[18]  Karl Henrik Johansson,et al.  Optimal stopping for event-triggered sensing and actuation , 2008, 2008 47th IEEE Conference on Decision and Control.

[19]  Bart De Schutter,et al.  Approximate dynamic programming with a fuzzy parameterization , 2010, Autom..

[20]  Paulo Tabuada,et al.  Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks , 2007, IEEE Transactions on Automatic Control.

[21]  Paulo Tabuada,et al.  To Sample or not to Sample: Self-Triggered Control for Nonlinear Systems , 2008, IEEE Transactions on Automatic Control.

[22]  Sandra Hirche,et al.  Optimal design of decentralized event-triggered controllers for large-scale systems with contention-based communication , 2011, IEEE Conference on Decision and Control and European Control Conference.

[23]  Dimos V. Dimarogonas,et al.  Novel event-triggered strategies for Model Predictive Controllers , 2011, IEEE Conference on Decision and Control and European Control Conference.

[24]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[25]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[26]  Wpmh Maurice Heemels,et al.  Self-triggered MPC for constrained linear systems and quadratic costs , 2012 .

[27]  M. Velasco,et al.  The Self Triggered Task Model for Real-Time Control Systems , 2003 .

[28]  Rémi Munos,et al.  Open Loop Optimistic Planning , 2010, COLT.

[29]  João Pedro Hespanha,et al.  A Survey of Recent Results in Networked Control Systems , 2007, Proceedings of the IEEE.