Actor-Critic Reinforcement Learning Control of Non-Strict Feedback Nonaffine Dynamic Systems

The most focuses of the existing actor-critic reinforcement learning control (ARLC) are on dealing with continuous affine systems or discrete nonaffine systems. In this paper, I propose a new ARLC method for continuous nonaffine dynamic systems subject to unknown dynamics and external disturbances. A new input-to-state stable system is developed to establish an augmented dynamic system, from which I further get a strict-feedback affine model that is convenient for control designing based on a model transformation approach. The Nussbaum function is connected with a fuzzy approximation to devise an actor network whose tracking performance is further enhanced via strengthening signals generated by a fuzzy critic network. The stability of the closed-loop control system is guaranteed by the Lyapunov synthesis. Finally, the comparison simulation results are presented to verify the design.

[1]  Qiong Wang,et al.  Concise Neural Nonaffine Control of Air-Breathing Hypersonic Vehicles Subject to Parametric Uncertainties , 2017 .

[2]  Jiaqi Huang,et al.  Guaranteeing preselected tracking quality for air-breathing hypersonic non-affine models with an unknown control direction via concise neural control , 2016, J. Frankl. Inst..

[3]  S. R. Nekoo,et al.  Finite-time state-dependent Riccati equation for time-varying nonaffine systems: rigid and flexible joint manipulator control. , 2015, ISA transactions.

[4]  Shaocheng Tong,et al.  Adaptive Neural Networks Prescribed Performance Control Design for Switched Interconnected Uncertain Nonlinear Systems , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Haibo He,et al.  Air-Breathing Hypersonic Vehicle Tracking Control Based on Adaptive Dynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Barjeev Tyagi,et al.  Adaptive Critic Design Using Policy Iteration Technique for LTI Systems: A Comprehensive Performance Analysis , 2015, Journal of Control, Automation and Electrical Systems.

[7]  Xiangwei Bu,et al.  Guaranteeing prescribed performance for air-breathing hypersonic vehicles via an adaptive non-affine tracking controller , 2018, Acta Astronautica.

[8]  Haibo He,et al.  Adaptive Critic Nonlinear Robust Control: A Survey , 2017, IEEE Transactions on Cybernetics.

[9]  Haibo He,et al.  Adaptive Learning and Control for MIMO System Based on Adaptive Dynamic Programming , 2011, IEEE Transactions on Neural Networks.

[10]  Xiangwei Bu,et al.  Guaranteeing prescribed output tracking performance for air-breathing hypersonic vehicles via non-affine back-stepping control design , 2017 .

[11]  Frank L. Lewis,et al.  Adaptive critic design using non‐linear network structures , 2003 .

[12]  Qiuye Sun,et al.  Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics , 2015, Neurocomputing.

[13]  Xiangwei Bu,et al.  Air-Breathing Hypersonic Vehicles Funnel Control Using Neural Approximation of Non-affine Dynamics , 2018, IEEE/ASME Transactions on Mechatronics.

[14]  Zhong-Ping Jiang,et al.  Adaptive dynamic programming and optimal control of nonlinear nonaffine systems , 2014, Autom..

[15]  Robert F. Stengel,et al.  Online Adaptive Critic Flight Control , 2004 .

[16]  Frank L. Lewis,et al.  Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Convergence Analysis , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  Chuan-Kai Lin.  Robust adaptive critic control of nonlinear systems using fuzzy basis function networks: An LMI approach , 2007, Inf. Sci..

[18]  Qinglai Wei,et al.  Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming , 2012, Autom..

[19]  Xiangwei Bu,et al.  A new prescribed performance control approach for uncertain nonlinear dynamic systems via back-stepping , 2018, J. Frankl. Inst..

[20]  Chuan-Kai Lin,et al.  Adaptive critic autopilot design of Bank-to-turn missiles using fuzzy basis function networks , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Shuzhi Sam Ge,et al.  A direct method for robust adaptive nonlinear control with guaranteed transient performance , 1999 .

[22]  Youxian Sun,et al.  Adaptive neural control of high-order uncertain nonaffine systems: A transformation to affine systems approach , 2014, Autom..

[23]  Wee Chin Wong,et al.  Approximate dynamic programming approach for process control , 2009 .

[24]  Wei-Song Lin,et al.  Adaptive critic motion control design of autonomous wheeled mobile robot by dual heuristic programming , 2008, Autom..

[25]  Shaocheng Tong,et al.  Adaptive Fuzzy Tracking Control Design for SISO Uncertain Nonstrict Feedback Nonlinear Systems , 2016, IEEE Transactions on Fuzzy Systems.

[26]  Derong Liu,et al.  Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning , 2014, Neural Networks.

[27]  Niket S. Kaisare,et al.  Approximate dynamic programming based control of hyperbolic PDE systems using reduced-order models from method of characteristics , 2013, Comput. Chem. Eng..

[28]  Qiuye Sun,et al.  Adaptive dynamic programming-based optimal control of unknown nonaffine nonlinear discrete-time systems with proof of convergence , 2012, Neurocomputing.

[29]  Yun Zhang,et al.  Neural Network Learning and Robust Stabilization of Nonlinear Systems With Dynamic Uncertainties , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Rui Zhang,et al.  Tracking differentiator design for the robust backstepping control of a flexible air-breathing hypersonic vehicle , 2015, J. Frankl. Inst..

[31]  Zhongke Shi,et al.  DOB-Based Neural Control of Flexible Hypersonic Flight Vehicle Considering Wind Effects , 2017, IEEE Transactions on Industrial Electronics.