An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs

In this paper, the adaptive dynamic programming (ADP) approach is employed for designing an optimal controller of unknown discrete-time nonlinear systems with control constraints. A neural network is constructed for identifying the unknown dynamical system with stability proof. Then, the iterative ADP algorithm is developed to solve the optimal control problem with convergence analysis. Two other neural networks are introduced for approximating the cost function and its derivatives and the control law, under the framework of globalized dual heuristic programming technique. Furthermore, two simulation examples are included to verify the theoretical results.

[1]  Zhihong Man,et al.  A new robust training algorithm for a class of single-hidden layer feedforward neural networks , 2011, Neurocomputing.

[2]  Dianhui Wang,et al.  Global Convergence of Online BP Training With Dynamic Learning Rate , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[3]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[4]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[5]  J. Sarangapani.  Neural Network Control of Nonlinear Discrete-Time Systems (Public Administration and Public Policy) , 2006 .

[6]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Hsin-Yi Lin,et al.  Self-organizing state aggregation for architecture design of Q-learning , 2011, Inf. Sci..

[8]  Dimitri P. Bertsekas,et al.  Temporal Difference Methods for General Projected Equations , 2011, IEEE Transactions on Automatic Control.

[9]  Jagannathan Sarangapani,et al.  Neural Network Control of Nonlinear Discrete-Time Systems , 2018 .

[10]  Leang-San Shieh,et al.  A new approach for neural control of nonlinear discrete dynamic systems , 2005, Inf. Sci..

[11]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[12]  Stefan Preitl,et al.  Iterative Feedback Tuning in Fuzzy Control Systems. Theory and Applications , 2006 .

[13]  Luigi Fortuna,et al.  Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control , 2009 .

[14]  D. Liu,et al.  Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With $\varepsilon$-Error Bound , 2011, IEEE Transactions on Neural Networks.

[15]  Sarangapani Jagannathan,et al.  Optimal control of unknown affine nonlinear discrete-time systems using offline-trained neural networks with proof of convergence , 2009, Neural Networks.

[16]  Jennie Si,et al.  Online learning control by association and reinforcement , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[17]  Xuesong Wang,et al.  A fuzzy Actor-Critic reinforcement learning network , 2007, Inf. Sci..

[18]  Andrew Lim,et al.  Example-based learning particle swarm optimization for continuous optimization , 2012, Information Sciences.

[19]  Stefan Preitl,et al.  Application of IFT and SPSA to Servo System Control , 2011, IEEE Transactions on Neural Networks.

[20]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[21]  Long Li,et al.  A modified gradient-based neuro-fuzzy learning algorithm and its convergence , 2010, Inf. Sci..

[22]  Frank L. Lewis,et al.  Model-free H∞ control design for unknown linear discrete-time systems via Q-learning with LMI , 2010, Autom..

[23]  Kumpati S. Narendra,et al.  Control of nonlinear dynamical systems using neural networks: controllability and stabilization , 1993, IEEE Trans. Neural Networks.

[24]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[25]  Jennie Si,et al.  Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence) , 2004 .

[26]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[27]  B. C. Brookes,et al.  Information Sciences , 2020, Cognitive Skills You Need for the 21st Century.

[28]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[29]  Sergio M. Savaresi,et al.  Direct nonlinear control design: the virtual reference feedback tuning (VRFT) approach , 2006, IEEE Transactions on Automatic Control.

[30]  Derong Liu,et al.  Optimal Control for a Class of Unknown Nonlinear Systems via the Iterative GDHP Algorithm , 2011, ISNN.

[31]  S. N. Balakrishnan,et al.  Approximate dynamic programming solutions with a single network adaptive critic for a class of nonlinear systems , 2011 .

[32]  Witold Pedrycz,et al.  Type-2 fuzzy neural networks with fuzzy clustering and differential evolution optimization , 2011, Inf. Sci..

[33]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[34]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[35]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.

[36]  Chuan-Kai Lin.  Robust adaptive critic control of nonlinear systems using fuzzy basis function networks: An LMI approach , 2007, Inf. Sci..

[37]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[38]  Derong Liu,et al.  Adaptive dynamic programming for finite-horizon optimal tracking control of a class of nonlinear systems , 2011, Proceedings of the 30th Chinese Control Conference.