Off-Policy Integral Reinforcement Learning for Semi-Global Constrained Output Regulation of Continuous-Time Linear Systems

This paper presents a data-driven method based on off-policy integral reinforcement learning to solve the semi-global output regulation of continuous-time linear systems with input saturation. A family of state feedback laws for the input constrained output regulation problem is designed based on solving an algebraic Riccati equation. In contrast to the existing methods, complete knowledge of the system dynamics is no longer required in this paper. Instead, the data collected from online implementation is efficiently utilized to design the controller. Therefore, the controller design in this paper is data-driven. It is shown that the presented method can find feedback control inputs with constraint of amplitude saturation and stabilize a given linear system with all its poles inside or on the imaginary axis. Finally, a simulation example is conducted to show the validity of the presented approach to solve the semi-global output regulation of continuous-time linear systems with input saturation.

[1]  A. Isidori,et al.  Output regulation of nonlinear systems , 1990 .

[2]  Jie Huang,et al.  Nonlinear Output Regulation: Theory and Applications , 2004 .

[3]  Yixin Yin,et al.  Optimal Containment Control of Unknown Heterogeneous Systems With Active Leaders , 2019, IEEE Transactions on Control Systems Technology.

[4]  B. Francis.  The linear multivariable regulator problem , 1976, 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes.

[5]  James Lam,et al.  Semiglobal Observer-Based Leader-Following Consensus With Input Saturation , 2014, IEEE Transactions on Industrial Electronics.

[6]  Frank L. Lewis,et al.  Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems , 2014, Autom..

[7]  Shangtai Jin,et al.  Data-Driven Model-Free Adaptive Control for a Class of MIMO Nonlinear Discrete-Time Systems , 2011, IEEE Transactions on Neural Networks.

[8]  Frank L. Lewis,et al.  Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics , 2014, Autom..

[9]  Zhong-Ping Jiang,et al.  Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[10]  Qinglai Wei,et al.  Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming , 2012, Autom..

[11]  Ali Saberi,et al.  Control of Linear Systems with Regulation and Input Constraints , 2000 .

[12]  Zongli Lin,et al.  Output regulation for linear systems subject to input saturation , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[13]  C. Byrnes.  Output Regulation of Uncertain Nonlinear Systems , 1997 .

[14]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[15]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[16]  Frank L. Lewis,et al.  Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Bruce A. Francis,et al.  The internal model principle of control theory , 1976, Autom..

[18]  Frank L. Lewis,et al.  Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning , 2014, IEEE Transactions on Automatic Control.

[19]  Yixin Yin,et al.  Off-policy reinforcement learning for robust control of discrete-time uncertain linear systems , 2017, 2017 36th Chinese Control Conference (CCC).

[20]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Yixin Yin,et al.  Leader–Follower Output Synchronization of Linear Heterogeneous Systems With Active Leader Using Reinforcement Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Guanrong Chen,et al.  Fully Distributed Event-Triggered Semiglobal Consensus of Multi-agent Systems With Input Saturation , 2017, IEEE Transactions on Industrial Electronics.

[23]  Zongli Lin,et al.  Output regulation for linear discrete-time systems subject to input saturation , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[24]  Frank L. Lewis,et al.  H∞ control of linear discrete-time systems: Off-policy reinforcement learning , 2017, Autom..

[25]  Frank L. Lewis,et al.  $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Warren E. Dixon,et al.  Approximate optimal trajectory tracking for continuous-time nonlinear systems , 2013, Autom..

[27]  Shangtai Jin,et al.  A Novel Data-Driven Control Approach for a Class of Discrete-Time Nonlinear Systems , 2011, IEEE Transactions on Control Systems Technology.

[28]  Yixin Yin,et al.  Hamiltonian-Driven Adaptive Dynamic Programming for Continuous Nonlinear Dynamical Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Frank L. Lewis,et al.  Optimal Control: Lewis/Optimal Control 3e , 2012 .

[30]  Tingwen Huang,et al.  Off-Policy Reinforcement Learning for $ H_\infty $ Control Design , 2013, IEEE Transactions on Cybernetics.