Decentralized adaptive tracking control scheme for nonlinear large-scale interconnected systems via adaptive dynamic programming

Abstract In this paper, the decentralized tracking problem for nonlinear large-scale interconnected systems is firstly transformed to optimal regulation problem for N augmented subsystems composed of the error system dynamics and the command generator dynamic associated with each isolated subsystems. The proposed novel formulation of decentralized adaptive tracking control strategy consists of a steady-state controller and a modified optimal feedback controller. Design parameters-dependent feasibility conditions are formulated by using Lyapunov theory to guarantee the existence of our proposed decentralized control scheme. A single critic neural network (NN)-based adaptive dynamic programming algorithm is used to find the estimation of optimal control policy, which is implemented online in real-time. By employing a stabilizing term in the critic NN weight updating law, there is no requirement for adopting initial admissible control in the proposed algorithm. Stability analysis of the closed-loop augmented subsystem is performed to show that all tracking errors and NN weight approximation errors are uniformly ultimately bounded (UUB). Furthermore, the approximated tracking control policy converges to the ideal control input with a small bounded error. Finally, the effectiveness of the proposed approach is demonstrated by some simulation results.

[1]  Kyriakos G. Vamvoudakis,et al.  Non-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems , 2015, Autom..

[2]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[3]  Lin Shi,et al.  Decentralized Adaptive Controller Design For Large-Scale Systems With Higher-Order Interconnections , 1992, 1992 American Control Conference.

[4]  Huaguang Zhang,et al.  Finite horizon optimal control of non-linear discrete-time switched systems using adaptive dynamic programming with ε-error bound , 2014, Int. J. Syst. Sci..

[5]  Derong Liu,et al.  Adaptive Dynamic Programming for Optimal Tracking Control of Unknown Nonlinear Systems With Application to Coal Gasification , 2014, IEEE Transactions on Automation Science and Engineering.

[6]  Chao Li,et al.  Neural-network-based decentralized control of continuous-time nonlinear interconnected systems with unknown dynamics , 2015, Neurocomputing.

[7]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[8]  Frank L. Lewis,et al.  Policy Iterations on the Hamilton–Jacobi–Isaacs Equation for $H_{\infty}$ State Feedback Control With Input Saturation , 2006, IEEE Transactions on Automatic Control.

[9]  Raymond A. DeCarlo,et al.  Decentralized tracking for a class of interconnected nonlinear systems using variable structure control , 1988, Autom..

[10]  S. Glad.  On the gain margin of nonlinear and optimal regulators , 1984, 1982 21st IEEE Conference on Decision and Control.

[11]  Feng Lin,et al.  Robust Control of Nonlinear Systems: Compensating for Uncertainty , 1990, 1990 American Control Conference.

[12]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Hyochoong Bang,et al.  Design and analysis of optimal controller for fuzzy systems with input constraint , 2004, IEEE Transactions on Fuzzy Systems.

[14]  Derong Liu,et al.  Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming , 2014, Inf. Sci..

[15]  P. Olver.  Nonlinear Systems , 2013 .

[16]  Youxian Sun,et al.  Robust Integral of Neural Network and Error Sign Control of MIMO Nonlinear Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[17]  M. Gopal,et al.  Fixed final time optimal control approach for bounded robust controller design using Hamilton-Jacobi-Bellman solution , 2009 .

[18]  Derong Liu,et al.  Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming , 2016, Neurocomputing.

[19]  Derong Liu,et al.  Adaptive Dynamic Programming for Control: Algorithms and Stability , 2012 .

[20]  Warren E. Dixon,et al.  Approximate optimal trajectory tracking for continuous-time nonlinear systems , 2013, Autom..

[21]  Huaguang Zhang,et al.  Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games via Fuzzy Adaptive Dynamic Programming , 2015, IEEE Transactions on Fuzzy Systems.

[22]  Hao Xu,et al.  Neural Network-Based Finite Horizon Stochastic Optimal Control Design for Nonlinear Networked Control Systems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Long Cheng,et al.  A Recurrent Neural Network for Hierarchical Control of Interconnected Dynamic Systems , 2007, IEEE Transactions on Neural Networks.

[24]  Derong Liu,et al.  Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[26]  Xing Xing,et al.  Robust adaptive decentralized control for a class of noaffine stochastic nonlinear interconnected systems , 2016, Neurocomputing.

[27]  Huaguang Zhang,et al.  Neural-Network-Based Constrained Optimal Control Scheme for Discrete-Time Switched Nonlinear System Using Dual Heuristic Programming , 2014, IEEE Transactions on Automation Science and Engineering.

[28]  Qiuye Sun,et al.  Adaptive dynamic programming-based optimal control of unknown nonaffine nonlinear discrete-time systems with proof of convergence , 2012, Neurocomputing.

[29]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[30]  Tingwen Huang,et al.  Off-Policy Reinforcement Learning for $ H_\infty $ Control Design , 2013, IEEE Transactions on Cybernetics.

[31]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[32]  Frank L. Lewis,et al.  Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances , 2016, IEEE Transactions on Cybernetics.

[33]  Frank L. Lewis,et al.  $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[34]  S. Jagannathan,et al.  Optimal control of affine nonlinear continuous-time systems using an online Hamilton-Jacobi-Isaacs formulation , 2010, 49th IEEE Conference on Decision and Control (CDC).

[35]  Warren E. Dixon,et al.  Model-based reinforcement learning for infinite-horizon approximate optimal tracking , 2014, 53rd IEEE Conference on Decision and Control.

[36]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[37]  Shaocheng Tong,et al.  Adaptive fuzzy decentralized tracking fault-tolerant control for stochastic nonlinear large-scale systems with unmodeled dynamics , 2014, Inf. Sci..

[38]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Huaguang Zhang,et al.  An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games , 2011, Autom..

[40]  Qinglai Wei,et al.  Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[41]  D. Hill,et al.  Nonlinear output stabilization control for multimachine power systems , 2000 .

[42]  Derong Liu,et al.  Infinite Horizon Self-Learning Optimal Control of Nonaffine Discrete-Time Nonlinear Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Zhong-Ping Jiang,et al.  Robust Adaptive Dynamic Programming for Large-Scale Systems With an Application to Multimachine Power Systems , 2012, IEEE Transactions on Circuits and Systems II: Express Briefs.

[44]  S Jagannathan,et al.  Power System Stabilization Using Adaptive Neural Network-Based Dynamic Surface Control , 2011, IEEE Transactions on Power Systems.

[45]  Derong Liu,et al.  Value Iteration Adaptive Dynamic Programming for Optimal Control of Discrete-Time Nonlinear Systems , 2016, IEEE Transactions on Cybernetics.

[46]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[47]  Ali Heydari,et al.  Fixed-final-time optimal tracking control of input-affine nonlinear systems , 2014, Neurocomputing.