Output feedback tracking control of a class of continuous nonlinear systems via adaptive dynamic programming approach

In this paper, an output feedback tracking control scheme is presented for a class of nonlinear systems via adaptive dynamic programming (ADP) technique. Observers are employed to reconstruct immeasurable information of the nonlinear systems, and by appropriate coordinate transformation, optimal tracking control issues are transformed into regulation problems, where critic neural network is constructed for the solution of Hamilton-Jacobi-Bellman (HJB) equation corresponding to tracking errors. It is proven that all signals in the closed-loop system are uniformly ultimately bounded by the Lyapunov approach. Finally, a simulation example is provided for illustrate the efficiency of the proposed scheme.

[1]  Warren B. Powell,et al.  An Optimal Approximate Dynamic Programming Algorithm for Concave, Scalar Storage Problems With Vector-Valued Controls , 2013, IEEE Transactions on Automatic Control.

[2]  Heidar Ali Talebi,et al.  A stable neural network-based observer with application to flexible-joint manipulators , 2006, IEEE Transactions on Neural Networks.

[3]  Derong Liu,et al.  Neural-Network-Based Online HJB Solution for Optimal Robust Guaranteed Cost Control of Continuous-Time Uncertain Nonlinear Systems , 2014, IEEE Transactions on Cybernetics.

[4]  Shin'ichi Tamura,et al.  Capabilities of a four-layered feedforward neural network: four layers versus three , 1997, IEEE Trans. Neural Networks.

[5]  Haibo He,et al.  Data-Driven Tracking Control With Adaptive Dynamic Programming for a Class of Continuous-Time Nonlinear Systems , 2017, IEEE Transactions on Cybernetics.

[6]  Haibo He,et al.  Adaptive Event-Triggered Control Based on Heuristic Dynamic Programming for Nonlinear Discrete-Time Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[9]  Daniel W. C. Ho,et al.  Optimal distributed stochastic mirror descent for strongly convex optimization , 2016, Autom..

[10]  Derong Liu,et al.  Adaptive Dynamic Programming for Optimal Tracking Control of Unknown Nonlinear Systems With Application to Coal Gasification , 2014, IEEE Transactions on Automation Science and Engineering.

[11]  Sarangapani Jagannathan,et al.  Output Feedback Control of a Quadrotor UAV Using Neural Networks , 2010, IEEE Transactions on Neural Networks.

[12]  Frank L. Lewis,et al.  Neurodynamic Programming and Zero-Sum Games for Constrained Control Systems , 2008, IEEE Transactions on Neural Networks.

[13]  Derong Liu,et al.  Manifold Regularized Reinforcement Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[15]  Haibo He,et al.  Optimal Control for Unknown Discrete-Time Nonlinear Markov Jump Systems Using Adaptive Dynamic Programming , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Derong Liu,et al.  Observer based adaptive dynamic programming for fault tolerant control of a class of nonlinear systems , 2017, Inf. Sci..

[17]  K. Vamvoudakis.  Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[18]  Randal W. Beard,et al.  Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation , 1997, Autom..

[19]  Haibo He,et al.  Online Learning Control Using Adaptive Critic Designs With Sparse Kernel Machines , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[21]  Frank L. Lewis,et al.  Multimodel neural networks identification and failure detection of nonlinear systems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[22]  Frank L. Lewis,et al.  Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2010, Autom..

[23]  Haibo He,et al.  Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints , 2017, IEEE Trans. Neural Networks Learn. Syst..

[24]  Yuzhu Huang,et al.  Bounded robust control design for uncertain nonlinear systems using single-network adaptive dynamic programming , 2017, Neurocomputing.

[25]  Huaguang Zhang,et al.  Optimal Output Regulation for Heterogeneous Multiagent Systems via Adaptive Dynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Indra Narayan Kar,et al.  Bounded robust control of nonlinear systems using neural network–based HJB solution , 2011, IEEE Transactions on Automation Science and Engineering.

[27]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[28]  Tingwen Huang,et al.  Multi-step heuristic dynamic programming for optimal control of nonlinear discrete-time systems , 2017, Inf. Sci..

[29]  Frank L. Lewis,et al.  Mixed Iterative Adaptive Dynamic Programming for Optimal Battery Energy Control in Smart Residential Microgrids , 2017, IEEE Transactions on Industrial Electronics.

[30]  Derong Liu,et al.  Adaptive Dynamic Programming for Control: Algorithms and Stability , 2012 .

[31]  Shaocheng Tong,et al.  Fuzzy Adaptive Output Feedback Optimal Control Design for Strict-Feedback Nonlinear Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[32]  Haibo He,et al.  An Event-Triggered ADP Control Approach for Continuous-Time System With Unknown Internal States , 2017, IEEE Transactions on Cybernetics.

[33]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[34]  Derong Liu,et al.  Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints , 2015, IEEE Transactions on Cybernetics.

[35]  Warren E. Dixon,et al.  Approximate optimal trajectory tracking for continuous-time nonlinear systems , 2013, Autom..

[36]  Derong Liu,et al.  An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs , 2013, Inf. Sci..

[37]  Derong Liu,et al.  Data-based robust adaptive control for a class of unknown nonlinear constrained-input systems via integral reinforcement learning , 2016, Inf. Sci..

[38]  Xiaohong Cui,et al.  Online finite-horizon optimal learning algorithm for nonzero-sum games with partially unknown dynamics and constrained inputs , 2016, Neurocomputing.

[39]  Daniel W. C. Ho,et al.  Clustered Event-Triggered Consensus Analysis: An Impulsive Framework , 2016, IEEE Transactions on Industrial Electronics.

[40]  Frank L. Lewis,et al.  Neural Network Control Of Robot Manipulators And Non-Linear Systems , 1998 .

[41]  Xiong Yang,et al.  Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints , 2014, Int. J. Control.

[42]  Avimanyu Sahoo,et al.  Stochastic Optimal Regulation of Nonlinear Networked Control Systems by Using Event-Driven Adaptive Dynamic Programming , 2017, IEEE Transactions on Cybernetics.

[43]  Zhong-Ping Jiang,et al.  Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[44]  Yang Xiong,et al.  Adaptive Dynamic Programming with Applications in Optimal Control , 2017 .

[45]  Derong Liu,et al.  Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming , 2013, Int. J. Control.

[46]  Yu Jiang,et al.  Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[48]  Xiaohong Cui,et al.  Data-based approximate optimal control for nonzero-sum games of multi-player systems using adaptive dynamic programming , 2018, Neurocomputing.

[49]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[50]  F. Lewis,et al.  Model-free Q-learning designs for discrete-time zero-sum games with application to H-infinity control , 2007, 2007 European Control Conference (ECC).

[51]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.