Data-Driven Adaptive Optimal Tracking Control for Completely Unknown Systems

In this paper, an online data-driven based solution is developed for linear quadratic tracking (LQT) problem of linear systems with completely unknown dynamics. By applying the vectorization operator and Kronecker product, an adaptive identifier is first built to identify the unknown system dynamics, where a new adaptive law with guaranteed convergence is proposed. By using system augmentation method and introducing a discounted factor in the cost function, a compact form of LQT formulation is proposed, where the feedforward and feedback control actions can be obtained simultaneously. Finally, a new policy iteration is introduced to solve the derived augmented algebraic Riccati equation (ARE). Simulation results are presented to demonstrate the effectiveness of the proposed algorithm.

[1]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[2]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[3]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[4]  Huaguang Zhang,et al.  Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming , 2014, Int. J. Control.

[5]  Derong Liu,et al.  Adaptive Dynamic Programming for Control , 2012 .

[6]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.

[7]  Jing Na,et al.  Adaptive optimal tracking control for continuous-time systems using identifier-critic based dynamic programming , 2017, 2017 36th Chinese Control Conference (CCC).

[8]  Derong Liu,et al.  Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach , 2012, Neurocomputing.

[9]  Frank L. Lewis,et al.  Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning , 2014, IEEE Transactions on Automatic Control.

[10]  J. Murray,et al.  The Adaptive Dynamic Programming Theorem , 2003 .

[11]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[12]  Frank L. Lewis,et al.  Adaptive optimal control for continuous-time linear systems based on policy iteration , 2009, Autom..

[13]  Yanhong Luo,et al.  Data-driven optimal tracking control for a class of affine non-linear continuous-time systems with completely unknown dynamics , 2016 .

[14]  Guido Herrmann,et al.  Adaptive optimal tracking control of unknown nonlinear systems using system augmentation , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[15]  Frank L. Lewis,et al.  Actor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Sumit Kumar Jha,et al.  Data-Driven Adaptive LQR for Completely Unknown LTI Systems , 2017 .

[17]  Frank L. Lewis,et al.  Reinforcement Learning and Approximate Dynamic Programming for Feedback Control , 2012 .

[18]  J. Na,et al.  Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.