Observer-critic structure-based adaptive dynamic programming for decentralised tracking control of unknown large-scale nonlinear systems

ABSTRACT In this paper, a decentralised tracking control (DTC) scheme is developed for unknown large-scale nonlinear systems by using observer-critic structure-based adaptive dynamic programming. The control consists of local desired control, local tracking error control and a compensator. By introducing the local neural network observer, the subsystem dynamics can be identified. The identified subsystems can be used for the local desired control and the control input matrix, which is used in local tracking error control. Meanwhile, Hamiltonian-Jacobi-Bellman equation can be solved by constructing a critic neural network. Thus, the local tracking error control can be derived directly. To compensate the overall error caused by substitution, observation and approximation of the local tracking error control, an adaptive robustifying term is employed. Simulation examples are provided to demonstrate the effectiveness of the proposed DTC scheme.

[1]  Naceur Benhadj Braiek,et al.  H∞ optimization-based decentralized control of linear interconnected systems with nonlinear interconnections , 2014, J. Frankl. Inst..

[2]  Huaguang Zhang,et al.  Finite horizon optimal control of non-linear discrete-time switched systems using adaptive dynamic programming with ε-error bound , 2014, Int. J. Syst. Sci..

[3]  Derong Liu,et al.  Adaptive Dynamic Programming for Optimal Tracking Control of Unknown Nonlinear Systems With Application to Coal Gasification , 2014, IEEE Transactions on Automation Science and Engineering.

[4]  Zhong-Ping Jiang,et al.  Robust Adaptive Dynamic Programming , 2017 .

[5]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[6]  Derong Liu,et al.  Decentralized guaranteed cost control of interconnected systems with uncertainties: A learning-based optimal control strategy , 2016, Neurocomputing.

[7]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Jin Bae Park,et al.  Decentralized Sampled-Data Fuzzy Observer Design for Nonlinear Interconnected Systems , 2016, IEEE Transactions on Fuzzy Systems.

[9]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Derong Liu,et al.  Data-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[11]  Wei Xing Zheng,et al.  Decentralized tracking control for non-Gaussian large-scale interconnected distribution systems , 2013, Proceedings of the 32nd Chinese Control Conference.

[12]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[13]  Jennie Si,et al.  Helicopter trimming and tracking control using direct neural dynamic programming , 2003, IEEE Trans. Neural Networks.

[14]  Haibo He,et al.  Event-Driven Adaptive Robust Control of Nonlinear Systems With Uncertainties Through NDP Strategy , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Chao Li,et al.  Neural-network-based decentralized control of continuous-time nonlinear interconnected systems with unknown dynamics , 2015, Neurocomputing.

[16]  Derong Liu,et al.  Observer based adaptive dynamic programming for fault tolerant control of a class of nonlinear systems , 2017, Inf. Sci..

[17]  Zhong-Ping Jiang,et al.  Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming , 2016, Autom..

[18]  Aiguo Song,et al.  Iterative GDHP-based approximate optimal tracking control for a class of discrete-time nonlinear systems , 2016, Neurocomputing.

[19]  Derong Liu,et al.  Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming , 2016, Neurocomputing.

[20]  J. Bao,et al.  Decentralized Nonlinear Control of Process Networks Based on Dissipativity – A Hamilton-Jacobi Equation Approach , 2014 .

[21]  Sung Jin Yoo,et al.  Decentralized approximation-free control for uncertain large-scale pure-feedback systems with unknown time-delayed nonlinearities and control directions , 2016 .

[22]  Hui Zhang,et al.  Decentralized robust attitude tracking control for spacecraft networks under unknown inertia matrices , 2015, Neurocomputing.

[23]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[24]  Shaocheng Tong,et al.  Observed-Based Adaptive Fuzzy Decentralized Tracking Control for Switched Uncertain Nonlinear Large-Scale Systems With Dead Zones , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[25]  Bo Zhao,et al.  Local joint information based active fault tolerant control for reconfigurable manipulator , 2014 .

[26]  Andreas Kugi,et al.  Infinite-dimensional decentralized damping control of large-scale manipulators with hydraulic actuation , 2016, Autom..

[27]  Romeo Ortega,et al.  An Observer-Based Scheme for Decentralized Stabilization of Large-Scale Systems With Application to Power Systems , 2015 .

[28]  Huaguang Zhang,et al.  Optimal Tracking Control for a Class of Nonlinear Discrete-Time Systems With Time Delays Based on Heuristic Dynamic Programming , 2011, IEEE Transactions on Neural Networks.

[29]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[30]  Chao Lu,et al.  Direct Heuristic Dynamic Programming for Damping Oscillations in a Large Power System , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Xin-Ping Guan,et al.  Decentralized Output Feedback Adaptive NN Tracking Control for Time-Delay Stochastic Nonlinear Systems With Prescribed Performance , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[32]  John N. Tsitsiklis,et al.  Neuro-dynamic programming: an overview , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[33]  Sarangapani Jagannathan,et al.  Decentralized Optimal Control of a Class of Interconnected Nonlinear Discrete-Time Systems by Using Online Hamilton-Jacobi-Bellman Formulation , 2011, IEEE Transactions on Neural Networks.

[34]  Derong Liu,et al.  Online fault compensation control based on policy iteration algorithm for a class of affine non-linear systems with actuator failures , 2016 .

[35]  Derong Liu,et al.  Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Kwang Y. Lee,et al.  An optimal tracking neuro-controller for nonlinear dynamic systems , 1996, IEEE Trans. Neural Networks.

[37]  Alireza Karimi,et al.  Fixed-order decentralized/distributed control of islanded inverter-interfaced microgrids , 2015 .

[38]  Zhong-Ping Jiang,et al.  Robust Adaptive Dynamic Programming for Large-Scale Systems With an Application to Multimachine Power Systems , 2012, IEEE Transactions on Circuits and Systems II: Express Briefs.

[39]  George W. Irwin,et al.  Stability Analysis and Implementation of a Decentralized Formation Control Strategy for Unmanned Vehicles , 2014, IEEE Transactions on Control Systems Technology.

[40]  Boris Lohmann,et al.  Output feedback decentralized control of large-scale systems using weighted sensitivity functions minimization , 2002, Syst. Control. Lett..

[41]  Qinglai Wei,et al.  A novel optimal tracking control scheme for a class of discrete-time nonlinear systems using generalised policy iteration adaptive dynamic programming algorithm , 2017, Int. J. Syst. Sci..

[42]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[43]  Derong Liu,et al.  Event-based input-constrained nonlinear H∞ state feedback with adaptive critic and neural implementation , 2016, Neurocomputing.

[44]  Changchun Hua,et al.  Decentralised fault-tolerant finite-time control for a class of interconnected non-linear systems , 2015 .

[45]  Jun Fu,et al.  Robust Adaptive Dynamic Programming of Two-Player Zero-Sum Games for Continuous-Time Linear Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Zhong-Ping Jiang,et al.  Decentralized Adaptive Optimal Control of Large-Scale Systems With Application to Power Systems , 2015, IEEE Transactions on Industrial Electronics.

[47]  Xiong Yang,et al.  Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints , 2014, Int. J. Control.

[48]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.