Integral reinforcement learning based decentralized optimal tracking control of unknown nonlinear large-scale interconnected systems with constrained-input

Abstract This paper deals with the decentralized optimal tracking control problem of large-scale interconnected systems with constrained-input. The large-scale interconnected systems are firstly transformed to several nominal isolated subsystems. Then, nominal isolated subsystems tracking problem is solved via integral reinforcement learning (IRL) method. It is proved that the solved optimal controllers ensure the boundedness of the original systems tracking error. The actor-critic neural network (NN) technique is used to approximate the critic cost and control policy to implement the IRL algorithm. The least squares approach is employed to solve the weights of actor-critic NN by using only system data. A simulation example is provided to verify the effectiveness of the controllers by comparing with the controllers without considering constrained-input.

[1]  Xiaohong Cui,et al.  Fault-tolerant optimised tracking control for unknown discrete-time linear systems using a combined reinforcement learning and residual compensation methodology , 2017, Int. J. Syst. Sci..

[2]  Huaguang Zhang,et al.  Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games via Fuzzy Adaptive Dynamic Programming , 2015, IEEE Transactions on Fuzzy Systems.

[3]  Chao Li,et al.  Neural-network-based decentralized control of continuous-time nonlinear interconnected systems with unknown dynamics , 2015, Neurocomputing.

[4]  Huaguang Zhang,et al.  Decentralized adaptive tracking control scheme for nonlinear large-scale interconnected systems via adaptive dynamic programming , 2017, Neurocomputing.

[5]  Derong Liu,et al.  Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Derong Liu,et al.  Data-based robust adaptive control for a class of unknown nonlinear constrained-input systems via integral reinforcement learning , 2016, Inf. Sci..

[7]  Josep M. Guerrero,et al.  A Multiagent-Based Consensus Algorithm for Distributed Coordinated Control of Distributed Generators in the Energy Internet , 2015, IEEE Transactions on Smart Grid.

[8]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[9]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[10]  Haibo He,et al.  A Novel Energy Function-Based Stability Evaluation and Nonlinear Control Approach for Energy Internet , 2017, IEEE Transactions on Smart Grid.

[11]  Zhong-Ping Jiang,et al.  Decentralized Adaptive Optimal Control of Large-Scale Systems With Application to Power Systems , 2015, IEEE Transactions on Industrial Electronics.

[12]  Qiuye Sun,et al.  Quasi-Z-Source Network-Based Hybrid Power Supply System for Aluminum Electrolysis Industry , 2017, IEEE Transactions on Industrial Informatics.

[13]  Yanhong Luo,et al.  Data-driven optimal tracking control for a class of affine non-linear continuous-time systems with completely unknown dynamics , 2016 .

[14]  Qiuye Sun,et al.  Optimal Placement of Energy Storage Devices in Microgrids via Structure Preserving Energy Function , 2016, IEEE Transactions on Industrial Informatics.

[15]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[16]  Yanan Li,et al.  Haptic Identification by ELM-Controlled Uncertain Manipulator , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  B. Paden,et al.  Nonlinear inversion-based output tracking , 1996, IEEE Trans. Autom. Control..

[18]  Qinglai Wei,et al.  Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Tingwen Huang,et al.  Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design , 2014, Autom..

[20]  J. Na,et al.  Online adaptive approximate optimal tracking control with simplified dual approximation structure for continuous-time unknown nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[21]  Yang Liu,et al.  ADP based optimal tracking control for a class of linear discrete-time system with multiple delays , 2016, Journal of the Franklin Institute.

[22]  I. Ha,et al.  Robust tracking in nonlinear systems , 1987 .

[23]  Derong Liu,et al.  Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming , 2016, Neurocomputing.

[24]  Derong Liu,et al.  Neuro-optimal tracking control for a class of discrete-time nonlinear systems via generalized value iteration adaptive dynamic programming approach , 2016, Soft Comput..

[25]  Derong Liu,et al.  Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics , 2012, Neural Computing and Applications.

[26]  Frank L. Lewis,et al.  Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems , 2014, Autom..

[27]  Liuqing Yang,et al.  Where does AlphaGo go: from church-turing thesis to AlphaGo thesis and beyond , 2016, IEEE/CAA Journal of Automatica Sinica.

[28]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[29]  E. van Kampen,et al.  Nonlinear Adaptive Flight Control Using Incremental Approximate Dynamic Programming and Output Feedback , 2017 .

[30]  Huaguang Zhang,et al.  Adaptive Fault-Tolerant Tracking Control for MIMO Discrete-Time Systems via Reinforcement Learning Algorithm With Less Learning Parameters , 2017, IEEE Transactions on Automation Science and Engineering.

[31]  Erik-Jan Van Kampen,et al.  Incremental model based online dual heuristic programming for nonlinear adaptive control , 2018 .

[32]  Shaocheng Tong,et al.  Neural Networks-Based Adaptive Finite-Time Fault-Tolerant Control for a Class of Strict-Feedback Switched Nonlinear Systems , 2019, IEEE Transactions on Cybernetics.

[33]  Xingjian Wang,et al.  Teleoperation Control Based on Combination of Wave Variable and Neural Networks , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[34]  Qinglai Wei,et al.  ADP-based optimal sensor scheduling for target tracking in energy harvesting wireless sensor networks , 2015, Neural Computing and Applications.

[35]  Kezhen Han,et al.  An integrated data-driven Markov parameters sequence identification and adaptive dynamic programming method to design fault-tolerant optimal tracking control for completely unknown model systems , 2017, J. Frankl. Inst..

[36]  Qinglai Wei,et al.  A novel optimal tracking control scheme for a class of discrete-time nonlinear systems using generalised policy iteration adaptive dynamic programming algorithm , 2017, Int. J. Syst. Sci..

[37]  Huaguang Zhang,et al.  Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming , 2014, Int. J. Control.

[38]  Li Li,et al.  Traffic signal timing via deep reinforcement learning , 2016, IEEE/CAA Journal of Automatica Sinica.

[39]  Huaguang Zhang,et al.  Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions , 2009, Neurocomputing.

[40]  Xiaohong Cui,et al.  Adaptive dynamic programming for H∞ tracking design of uncertain nonlinear systems with disturbances and input constraints , 2017 .