Adaptive optimal control for a class of uncertain systems with saturating actuators and external disturbance using integral reinforcement learning

The problem of adaptive optimal control for a class of nonlinear uncertain systems with saturating actuators and external disturbance is investigated in this paper. Considering the saturating actuators, a non-quadratic cost function is adopted. The key of this optimal control problem is to find the solution to the Hamilton Jacobi Bellman equation (HJB). An online intergral reinforcement learning (IRL) algorithm based-Neural Network (NN) is given to approximate the solution. Unlike traditional integral reinforcement learning algorithms, data onto a period of time stored together with current data are used to update the neural network weights in place of persistence of excitation (PE) condition. This method overcomes the shortcomings of the PE condition which is not easy to be checked online. Finally, numerical examples are given to show the effectiveness of the proposed methods.

[1]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[2]  Zhong-Ping Jiang,et al.  Global Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems , 2013, IEEE Transactions on Automatic Control.

[3]  Wei Lin,et al.  Global L 2-gain design for a class of nonlinear systems 1 1 Supported in part by NSF under grant ECS-9412340 and by MURST. , 1998 .

[4]  Zongli Lin,et al.  Robust stability analysis and fuzzy-scheduling control for nonlinear systems subject to actuator saturation , 2003, IEEE Trans. Fuzzy Syst..

[5]  Shaocheng Tong,et al.  Fuzzy Adaptive Output Feedback Optimal Control Design for Strict-Feedback Nonlinear Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[6]  Tingwen Huang,et al.  Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design , 2014, Autom..

[7]  Rastko R. Selmic,et al.  Neural network control of a class of nonlinear systems with actuator saturation , 2004 .

[8]  Frank L. Lewis,et al.  Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems , 2014, Autom..

[9]  B. Finlayson.  The method of weighted residuals and variational principles : with application in fluid mechanics, heat and mass transfer , 1972 .

[10]  A. Schaft.  L/sub 2/-gain analysis of nonlinear systems and nonlinear state-feedback H/sub infinity / control , 1992 .

[11]  Frank L. Lewis,et al.  Adaptive optimal control for continuous-time linear systems based on policy iteration , 2009, Autom..

[12]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[13]  T. Basar,et al.  H∞-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach , 1996, IEEE Trans. Autom. Control..

[14]  Tianyou Chai,et al.  Online Solution of Two-Player Zero-Sum Games for Continuous-Time Nonlinear Systems With Completely Unknown Dynamics , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[16]  G. Zames.  Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inverses , 1981 .

[17]  Frank L. Lewis,et al.  A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems , 2013, Autom..

[19]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[20]  Shuzhi Sam Ge,et al.  Robust Adaptive Neural Network Control for a Class of Uncertain MIMO Nonlinear Systems With Input Nonlinearities , 2010, IEEE Transactions on Neural Networks.

[21]  S. Lyshevski.  Optimal control of nonlinear continuous-time systems: design of bounded controllers via generalized nonquadratic functionals , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[22]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[23]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[24]  Randal W. Beard,et al.  Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation , 1997, Autom..

[25]  Richard Colgren.  Nonlinear H(Infinity) Control Of A Uav , 2004 .

[26]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[27]  Girish Chowdhary,et al.  Concurrent learning for convergence in adaptive control without persistency of excitation , 2010, 49th IEEE Conference on Decision and Control (CDC).