Optimal Control of Nonlinear Continuous-Time Systems in Strict-Feedback Form

This paper proposes a novel optimal tracking control scheme for nonlinear continuous-time systems in strict-feedback form with uncertain dynamics. The optimal tracking problem is transformed into an equivalent optimal regulation problem through a feedforward adaptive control input that is generated by modifying the standard backstepping technique. Subsequently, a neural network-based optimal control scheme is introduced to estimate the cost, or value function, over an infinite horizon for the resulting nonlinear continuous-time systems in affine form when the internal dynamics are unknown. The estimated cost function is then used to obtain the optimal feedback control input; therefore, the overall optimal control input for the nonlinear continuous-time system in strict-feedback form includes the feedforward plus the optimal feedback terms. It is shown that the estimated cost function minimizes the Hamilton-Jacobi-Bellman estimation error in a forward-in-time manner without using any value or policy iterations. Finally, optimal output feedback control is introduced through the design of a suitable observer. Lyapunov theory is utilized to show the overall stability of the proposed schemes without requiring an initial admissible controller. Simulation examples are provided to validate the theoretical results.

[1]  Warren E. Dixon,et al.  Asymptotic tracking by a reinforcement learning-based adaptive critic controller , 2011 .

[2]  R. Mahony,et al.  Robust trajectory tracking for a scale model autonomous helicopter , 2004 .

[3]  Charalampos P. Bechlioulis,et al.  Adaptive control with guaranteed transient and steady state tracking error bounds for strict feedback systems , 2009, Autom..

[4]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[5]  Randal W. Bea.  Successive Galerkin approximation algorithms for nonlinear optimal and robust control , 1998 .

[6]  Miroslav Krstic,et al.  Optimal design of adaptive tracking controllers for non-linear systems , 1997, Autom..

[7]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[8]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[9]  John T. Wen,et al.  Improving the performance of stabilizing controls for nonlinear systems , 1996 .

[10]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[11]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.

[12]  Jagannathan Sarangapani,et al.  Neural Network Control of Nonlinear Discrete-Time Systems , 2018 .

[13]  T. Dierks,et al.  Optimal control of affine nonlinear discrete-time systems , 2009, 2009 17th Mediterranean Conference on Control and Automation.

[14]  Shuzhi Sam Ge,et al.  Direct adaptive NN control of a class of nonlinear systems , 2002, IEEE Trans. Neural Networks.

[15]  Dan Wang,et al.  Neural network-based adaptive dynamic surface control for a class of uncertain nonlinear systems in strict-feedback form , 2005, IEEE Transactions on Neural Networks.

[16]  Cong Wang,et al.  Learning from neural control , 2006, IEEE Transactions on Neural Networks.

[17]  Sarangapani Jagannathan,et al.  Optimal adaptive control of nonlinear continuous-time systems in strict feedback form with unknown internal dynamics , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[18]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  P. Olver.  Nonlinear Systems , 2013 .

[20]  Qinglai Wei,et al.  A Novel Iterative $\theta $-Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems , 2014, IEEE Transactions on Automation Science and Engineering.

[21]  Haibo He,et al.  Online Learning Control Using Adaptive Critic Designs With Sparse Kernel Machines , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Shuzhi Sam Ge,et al.  Adaptive neural network control for strict-feedback nonlinear systems using backstepping design , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[23]  Miroslav Krstic,et al.  Nonlinear and adaptive control de-sign , 1995 .

[24]  Jang-Hyun Park,et al.  Adaptive Neural Control for Strict-Feedback Nonlinear Systems Without Backstepping , 2009, IEEE Transactions on Neural Networks.

[25]  Sarangapani Jagannathan,et al.  State and output feedback-based adaptive optimal control of nonlinear continuous-time systems in strict feedback form , 2012, 2012 American Control Conference (ACC).

[26]  Sarangapani Jagannathan,et al.  Adaptive neural network‐based optimal control of nonlinear continuous‐time systems in strict‐feedback form , 2014 .

[27]  Hassan Zargarzadeh,et al.  Robust optimal control of uncertain nonaffine MIMO nonlinear discrete‐time systems with application to HCCI engines , 2012 .

[28]  Maria Adler,et al.  Stable Adaptive Systems , 2016 .

[29]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[30]  Hao Xu,et al.  Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses , 2012, Autom..

[31]  Draguna Vrabie,et al.  Adaptive optimal controllers based on Generalized Policy Iteration in a continuous-time framework , 2009, 2009 17th Mediterranean Conference on Control and Automation.

[32]  Derong Liu,et al.  Online Synchronous Approximate Optimal Learning Algorithm for Multi-Player Non-Zero-Sum Games With Unknown Dynamics , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[33]  M. Krstić,et al.  Optimal design of adaptive tracking controllers for nonlinear systems , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[34]  Frank L. Lewis,et al.  Neural Network Control Of Robot Manipulators And Non-Linear Systems , 1998 .

[35]  Jennie Si,et al.  Handbook of Learning and Approximate Dynamic Programming (IEEE Press Series on Computational Intelligence) , 2004 .

[36]  Peter Kuster,et al.  Nonlinear And Adaptive Control Design , 2016 .

[37]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[38]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[39]  Frank L. Lewis,et al.  Adaptive optimal control for continuous-time linear systems based on policy iteration , 2009, Autom..

[40]  Sarangapani Jagannathan,et al.  Online optimal control of nonlinear discrete-time systems using approximate dynamic programming , 2011 .