A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm

In this paper, we aim to solve the infinite-time optimal tracking control problem for a class of discrete-time nonlinear systems using the greedy heuristic dynamic programming (HDP) iteration algorithm. A new type of performance index is defined because the existing performance indexes are very difficult in solving this kind of tracking problem, if not impossible. Via system transformation, the optimal tracking problem is transformed into an optimal regulation problem, and then, the greedy HDP iteration algorithm is introduced to deal with the regulation problem with rigorous convergence analysis. Three neural networks are used to approximate the performance index, compute the optimal control policy, and model the nonlinear system for facilitating the implementation of the greedy HDP iteration algorithm. An example is given to demonstrate the validity of the proposed optimal tracking control scheme.

[1]  S. Marsili-Libelli.  Optimal design of PID regulators , 1981 .

[2]  I. Ha,et al.  Robust tracking in nonlinear systems , 1987 .

[3]  Z. Gajic,et al.  The successive approximation procedure for finite-time optimal control of bilinear systems , 1994, IEEE Trans. Autom. Control..

[4]  B. Paden,et al.  Nonlinear inversion-based output tracking , 1996, IEEE Trans. Autom. Control..

[5]  Kwang Y. Lee,et al.  An optimal tracking neuro-controller for nonlinear dynamic systems , 1996, IEEE Trans. Neural Networks.

[6]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[7]  R. Beard,et al.  Synthesis and experimental testing of a nonlinear optimal tracking controller , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[8]  Jie Huang.  An algorithm to solve the discrete HJI equation arising in the L2 gain optimization problem , 1999 .

[9]  Jennie Si,et al.  Online learning control by association and reinforcement , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[10]  Derong Liu,et al.  Action-dependent adaptive critic designs , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[11]  George G. Lendaris,et al.  Adaptive dynamic programming , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[12]  Stephen P. Banks,et al.  Nonlinear optimal tracking control with application to super-tankers for autopilot design , 2004, Autom..

[13]  Robert F. Stengel,et al.  Online Adaptive Critic Flight Control , 2004 .

[14]  Yi Zhang,et al.  A self-learning call admission control scheme for CDMA cellular networks , 2005, IEEE Transactions on Neural Networks.

[15]  Huaguang Zhang,et al.  A Neural Dynamic Programming Approach F or Learning Control O f Failure Avoidance Problems , 2005 .

[16]  G. Tang,et al.  Approximate Optimal Tracking Control for a Class of Nonlinear Systems with Disturbances , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[17]  Derong Liu,et al.  Discrete-Time Adaptive Dynamic Programming using Wavelet Basis Function Neural Networks , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[18]  S. Jagannathan,et al.  Online Reinforcement Learning Neural Network Controller Design for Nanomanipulation , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[19]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.