Linear optimal tracking control: An adaptive dynamic programming approach

This paper addresses the optimal output regulation problem of linear systems with unknown system dynamics. The exogenous signal is presumed to be generated by a continuous-time linear exosystem. Firstly, we formulate the linear optimal output regulation problem (LOORP). Then, we give an offline solution of LOORP to design the optimal static state-feedback servoregulator by solving an algebraic Riccati equation (ARE) and a regulator equation. Instead of solving these two equations directly, by using state, input and exogenous signals collected online, we employ an approximate/adaptive dynamic programming (ADP) technique to seek online approximations of above equations whereby we get the approximated optimal servoregulator. Rigorous stability analysis shows that the closed-loop linear system is exponentially stable. Also, the output of the plant asymptotically tracks the given reference. Simulation results demonstrate the effectiveness of the proposed approach.

[1]  Zhong-Ping Jiang,et al.  Small-gain theorem for ISS systems and applications , 1994, Math. Control. Signals Syst..

[2]  Yu Jiang,et al.  Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[4]  P. Olver.  Nonlinear Systems , 2013 .

[5]  Frank L. Lewis,et al.  Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics , 2014, Autom..

[6]  Zheng Fang,et al.  Adaptive integral backstepping control for a 3-DOF helicopter , 2012, 2012 IEEE International Conference on Information and Automation.

[7]  Jie Huang,et al.  Nonlinear Output Regulation: Theory and Applications , 2004 .

[8]  E. Davison.  The robust control of a servomechanism problem for linear time-invariant multivariable systems , 1976 .

[9]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[10]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  D. Kleinman.  On an iterative technique for Riccati equation computations , 1968 .

[12]  W. Wonham,et al.  The internal model principle for linear multivariable regulators , 1975 .

[13]  B. Francis.  The linear multivariable regulator problem , 1976, 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes.

[14]  Derong Liu,et al.  Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach , 2012, Neurocomputing.

[15]  Huaguang Zhang,et al.  Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming , 2014, Int. J. Control.

[16]  Zhong-Ping Jiang,et al.  Adaptive and optimal output feedback control of linear systems: An adaptive dynamic programming approach , 2014, Proceeding of the 11th World Congress on Intelligent Control and Automation.

[17]  Frank L. Lewis,et al.  Reinforcement Learning and Approximate Dynamic Programming for Feedback Control , 2012 .

[18]  Arthur J. Krener.  The Construction of Optimal Linear and Nonlinear Regulators , 1992 .

[19]  Frank L. Lewis,et al.  Optimal Control: Lewis/Optimal Control 3e , 2012 .

[20]  W. Dixon.  Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles , 2014 .

[21]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[22]  A. Saberi,et al.  On optimal output regulation for linear systems , 2003 .

[23]  Zhong-Ping Jiang,et al.  Robust adaptive dynamic programming for linear and nonlinear systems: An overview , 2013, Eur. J. Control.