Nearly finite-horizon optimal control for a class of nonaffine time-delay nonlinear systems based on adaptive dynamic programming

In this paper, a novel adaptive dynamic programming (ADP) algorithm is developed to solve the nearly optimal finite-horizon control problem for a class of deterministic nonaffine nonlinear time-delay systems. The idea is to use ADP technique to obtain the nearly optimal control which makes the optimal performance index function close to the greatest lower bound of all performance index functions within finite time. The proposed algorithm contains two cases with respective different initial iterations. In the first case, there exists control policy which makes arbitrary state of the system reach to zero in one time step. In the second case, there exists a control sequence which makes the system reach to zero in multiple time steps. The state updating is used to determine the optimal state. Convergence analysis of the performance index function is given. Furthermore, the relationship between the iteration steps and the length of the control sequence is presented. Two neural networks are used to approximate the performance index function and compute the optimal control policy for facilitating the implementation of ADP iteration algorithm. At last, two examples are used to demonstrate the effectiveness of the proposed ADP iteration algorithm.

[1]  Yuzhang Lin,et al.  Optimal control for a class of nonlinear systems with state delay based on Adaptive Dynamic Programming with ε-error bound , 2013, 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[2]  Huaguang Zhang,et al.  Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Derong Liu,et al.  An Optimal Control Scheme for a Class of Discrete-time Nonlinear Systems with Time Delays Using Adaptive Dynamic Programming , 2010 .

[4]  Huaguang Zhang,et al.  Optimal control laws for time-delay systems with saturating actuators based on heuristic dynamic programming , 2010, Neurocomputing.

[5]  F. L. Lewis,et al.  Neural-network predictive control for nonlinear dynamic systems with time-delay , 2003, IEEE Trans. Neural Networks.

[6]  Jun Wang,et al.  Robustness Analysis of Global Exponential Stability of Recurrent Neural Networks in the Presence of Time Delays and Random Disturbances , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Huaguang Zhang,et al.  An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games , 2011, Autom..

[8]  Huaguang Zhang,et al.  Multi-objective optimal control for a class of unknown nonlinear systems based on finite-approximation-error ADP algorithm , 2013, Neurocomputing.

[9]  Vladimir L. Kharitonov,et al.  Stability of Time-Delay Systems , 2003, Control Engineering.

[10]  D. Liu,et al.  Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With $\varepsilon$-Error Bound , 2011, IEEE Transactions on Neural Networks.

[11]  D. H. Chyung,et al.  Linear Optimal Systems with Time Delays , 1966 .

[12]  R. Bellman.  Dynamic programming. , 1957, Science.

[13]  S. Niculescu.  Delay Effects on Stability: A Robust Control Approach , 2001 .

[14]  Vu Ngoc Phat,et al.  Controllability of discrete-time systems with multiple delays on controls and states , 1989 .

[15]  Guo-Xing Wen,et al.  Adaptive Consensus Control for a Class of Nonlinear Multiagent Time-Delay Systems Using Neural Networks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Qiuye Sun,et al.  Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics , 2015, Neurocomputing.

[17]  Derong Liu,et al.  Neural-network-based adaptive optimal tracking control scheme for discrete-time nonlinear systems with approximation errors , 2013, Neurocomputing.

[18]  Dong Chyung,et al.  Discrete optimal systems with time delay , 1968 .

[19]  Huaguang Zhang,et al.  Model-free optimal control design for a class of linear discrete-time systems with multiple delays using adaptive dynamic programming , 2014, Neurocomputing.

[20]  Qinglai Wei,et al.  Multi-objective optimal control for a class of nonlinear time-delay systems via adaptive dynamic programming , 2013, Soft Comput..

[21]  F. Lewis,et al.  Discrete-time nonlinear HJB solution using Approximate dynamic programming: Convergence Proof , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[22]  Huaguang Zhang,et al.  An Optimal Control Scheme for a Class of Discrete-time Nonlinear Systems with Time Delays Using Adap , 2010 .

[23]  M. Malek-Zavarei,et al.  Time-Delay Systems: Analysis, Optimization and Applications , 1987 .

[24]  Shuhui Li,et al.  Artificial Neural Networks for Control of a Grid-Connected Rectifier/Inverter Under Disturbance, Dynamic and Power Converter Switching Conditions , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Dongbin Zhao,et al.  Full-range adaptive cruise control based on supervised adaptive dynamic programming , 2014, Neurocomputing.

[27]  Dong H. Chyung.  Controllability of linear systems with multiple delays in control , 1970 .

[28]  Huaguang Zhang,et al.  Optimal Tracking Control for a Class of Nonlinear Discrete-Time Systems With Time Delays Based on Heuristic Dynamic Programming , 2011, IEEE Transactions on Neural Networks.

[29]  Dong H. Chyung,et al.  On the controllability of linear systems with delay in control , 1970 .