The finite-horizon optimal control for a class of time-delay affine nonlinear system

In this paper, a new iteration algorithm is proposed to solve the finite-horizon optimal control problem for a class of time-delay affine nonlinear systems with known system dynamic. First, we prove that the algorithm is convergent as the iteration step increases. Then, a theorem is presented to demonstrate that the limit of the iteration performance index function satisfies discrete-time Hamilton–Jacobi–Bellman (DTHJB) equation, and the finite-horizon iteration algorithm is presented with satisfactory accuracy error. At last, two neural networks are used to approximate the iteration performance index function and the corresponding control policy. In simulation part, an example is given to demonstrate the effectiveness of the proposed iteration algorithm.

[1]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[2]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[3]  Huaguang Zhang,et al.  An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games , 2011, Autom..

[4]  Derong Liu,et al.  Adaptive Critic Learning Techniques for Engine Torque and Air–Fuel Ratio Control , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[6]  D. Liu,et al.  Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With $\varepsilon$-Error Bound , 2011, IEEE Transactions on Neural Networks.

[7]  Khadija Iqbal,et al.  An introduction , 1996, Neurobiology of Aging.

[8]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[9]  M. Malek-Zavarei,et al.  Time-Delay Systems: Analysis, Optimization and Applications , 1987 .

[10]  Huaguang Zhang,et al.  Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions , 2009, Neurocomputing.

[11]  Richard S. Sutton,et al.  A Menu of Designs for Reinforcement Learning Over Time , 1995 .

[12]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  George G. Lendaris,et al.  Adaptive dynamic programming , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[14]  Frank L. Lewis,et al.  Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ Control , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Paul J. Werbos,et al.  Building and Understanding Adaptive Systems: A Statistical/Numerical Approach to Factory Automation and Brain Research , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  Donald A. Sofge,et al.  Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches , 1992 .

[17]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .