Robust optimal control for time-delay systems with dynamic uncertainties via ADP

This paper considers a robust optimal control design for a class of nonlinear discrete-time systems with unknown time-varying delays and dynamic uncertainties. An iterative control strategy based on adaptive dynamic programming (ADP) has been proposed. Neural networks are applied to realize the state prediction, the control input estimation and the performance index function approximation. The estimated control input and performance index function are updated iteratively. Furthermore, it has been proven that the approximated performance index function can converge to the optimal solution of the Hamilton-Jacobia-Bellman (HJB) equation. Finally, the proposed algorithm has been conducted to a numerical simulation. The simulation results demonstrate the effectiveness of the new design.

[1]  Derong Liu,et al.  Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming , 2014, Inf. Sci..

[2]  Derong Liu,et al.  Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  F. Lewis,et al.  Model-free Q-learning designs for discrete-time zero-sum games with application to H-infinity control , 2007, 2007 European Control Conference (ECC).

[4]  Haibo He,et al.  Adaptive Event-Triggered Control Based on Heuristic Dynamic Programming for Nonlinear Discrete-Time Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Jennie Si,et al.  A New Powered Lower Limb Prosthesis Control Framework Based on Adaptive Dynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Derong Liu,et al.  An Optimal Control Scheme for a Class of Discrete-time Nonlinear Systems with Time Delays Using Adaptive Dynamic Programming , 2010 .

[7]  Jennie Si,et al.  Online learning control by association and reinforcement , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[8]  Tianyou Chai,et al.  Robust adaptive quadratic tracking control of continuous-time linear systems with unknown dynamics , 2015, 2015 American Control Conference (ACC).

[9]  George G. Lendaris,et al.  Adaptive dynamic programming , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[10]  Frank L. Lewis,et al.  Actor-Critic Off-Policy Learning for Optimal Control of Multiple-Model Discrete-Time Systems , 2018, IEEE Transactions on Cybernetics.

[11]  Changyin Sun,et al.  Neural Network Control of a Robotic Manipulator With Input Deadzone and Output Constraint , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[12]  Xi Chen,et al.  Model-free reinforcement learning for nonlinear zero-sum games with simultaneous explorations , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[13]  Haibo He,et al.  Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Changyin Sun,et al.  An Event-Triggered Approach for Load Frequency Control With Supplementary ADP , 2017, IEEE Transactions on Power Systems.

[15]  Hao Xu,et al.  Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses , 2012, Autom..

[16]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[17]  Qinglai Wei,et al.  Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Huaguang Zhang,et al.  An Optimal Control Scheme for a Class of Discrete-time Nonlinear Systems with Time Delays Using Adap , 2010 .

[19]  Donald E. Kirk,et al.  Optimal control theory : an introduction , 1970 .

[20]  Minwoo Lee,et al.  Faster reinforcement learning after pretraining deep networks to predict state dynamics , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[21]  Huaguang Zhang,et al.  Robust exponential stability analysis of neural networks with multiple time delays , 2007, Neurocomputing.

[22]  Zhong-Ping Jiang,et al.  Robust approximate dynamic programming and global stabilization with nonlinear dynamic uncertainties , 2011, IEEE Conference on Decision and Control and European Control Conference.

[23]  Changyin Sun,et al.  Neural Network Control of a Flexible Robotic Manipulator Using the Lumped Spring-Mass Model , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[24]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Ali Heydari,et al.  Finite-Horizon Control-Constrained Nonlinear Optimal Control Using Single Network Adaptive Critics , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Shuzhi Sam Ge,et al.  Robust Adaptive Neural Network Control for a Class of Uncertain MIMO Nonlinear Systems With Input Nonlinearities , 2010, IEEE Transactions on Neural Networks.

[27]  Yu Jiang,et al.  Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Haibo He,et al.  A three-network architecture for on-line learning and optimization based on adaptive dynamic programming , 2012, Neurocomputing.