Approximate Optimal Distributed Control of Nonlinear Interconnected Systems Using Event-Triggered Nonzero-Sum Games

In this paper, approximate optimal distributed control schemes for a class of nonlinear interconnected systems with strong interconnections are presented using continuous and event-sampled feedback information. The optimal control design is formulated as an $N$ -player nonzero-sum game where the control policies of the subsystems act as players. An approximate Nash equilibrium solution to the game, which is the solution to the coupled Hamilton–Jacobi equation, is obtained using the approximate dynamic programming-based approach. A critic neural network (NN) at each subsystem is utilized to approximate the Nash solution and novel event-sampling conditions, that are decentralized, are designed to asynchronously orchestrate the sampling and transmission of state vector at each subsystem. To ensure the local ultimate boundedness of the closed-loop system state and NN parameter estimation errors, a hybrid-learning scheme is introduced and the stability is guaranteed using Lyapunov-based stability analysis. Finally, implementation of the proposed event-based distributed control scheme for linear interconnected systems is discussed. For completeness, Zeno-free behavior of the event-sampled system is shown analytically and a numerical example is included to support the analytical results.

[1]  Derong Liu,et al.  Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Avimanyu Sahoo,et al.  Optimal sampling and regulation of uncertain interconnected linear continuous time systems , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[3]  Snehasis Mukhopadhyay,et al.  To communicate or not to communicate: A decision-theoretic approach to decentralized adaptive control , 2010, Proceedings of the 2010 American Control Conference.

[4]  Marcus Johnson,et al.  Approximate $N$ -Player Nonzero-Sum Game Solution for an Uncertain Continuous Nonlinear System , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Frank L. Lewis,et al.  Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  Dragoslav D. Šiljak,et al.  Decentralized control of complex systems , 2012 .

[8]  Xiaofeng Wang,et al.  Event-Triggering in Distributed Networked Control Systems , 2011, IEEE Transactions on Automatic Control.

[9]  Sarangapani Jagannathan,et al.  Approximate optimal distributed control of uncertain nonlinear interconnected systems with event-sampled feedback , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[10]  Karthikeyan Rajagopal,et al.  Neural Network-Based Solutions for Stochastic Optimal Control Using Path Integrals , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Sarangapani Jagannathan,et al.  A self-tuning optimal controller for affine nonlinear continuous-time systems with unknown internal dynamics , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[12]  Frank L. Lewis,et al.  Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton-Jacobi equations , 2011, Autom..

[13]  Ali Heydari,et al.  Stability Analysis of Optimal Adaptive Control Under Value Iteration Using a Stabilizing Initial Policy , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Haibo He,et al.  Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[16]  Wassim M. Haddad,et al.  Impulsive and Hybrid Dynamical Systems: Stability, Dissipativity, and Control , 2006 .

[17]  Hao Xu,et al.  Neural Network-Based Finite Horizon Stochastic Optimal Control Design for Nonlinear Networked Control Systems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Onder Tutsoy,et al.  Adaptive estimator design for unstable output error systems: A test problem and traditional system identification based analysis , 2015, J. Syst. Control. Eng..

[19]  Paulo Tabuada,et al.  Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks , 2007, IEEE Transactions on Automatic Control.

[20]  Sarangapani Jagannathan,et al.  Event-Triggered Distributed Approximate Optimal State and Output Control of Affine Nonlinear Interconnected Systems , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Zhong-Ping Jiang,et al.  A Small-Gain Approach to Robust Event-Triggered Control of Nonlinear Systems , 2015, IEEE Transactions on Automatic Control.

[22]  Frank L. Lewis,et al.  Optimal and Autonomous Control Using Reinforcement Learning: A Survey , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Sarangapani Jagannathan,et al.  Distributed Adaptive Optimal Regulation of Uncertain Large-Scale Linear Networked Control Systems Using Q-Learning , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[24]  Rahul Singh,et al.  Decentralized Control via Dynamic Stochastic Prices: The Independent System Operator Problem , 2016, IEEE Transactions on Automatic Control.

[25]  J. Nash.  NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[26]  Yusheng Liu,et al.  Decentralized robust adaptive control of nonlinear systems with unmodeled dynamics , 2002, IEEE Trans. Autom. Control..

[27]  Frank L. Lewis,et al.  Neural Network Control Of Robot Manipulators And Non-Linear Systems , 1998 .

[28]  Kok Kiong Tan,et al.  Decentralized control design for large-scale systems with strong interconnections using neural networks , 2003, IEEE Trans. Autom. Control..

[29]  Derong Liu,et al.  Neural-Network-Based Distributed Adaptive Robust Control for a Class of Nonlinear Multiagent Systems With Time Delays and External Noises , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[30]  Haibo He,et al.  Event-Driven Adaptive Robust Control of Nonlinear Systems With Uncertainties Through NDP Strategy , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Y. Ho,et al.  Nonzero-sum differential games , 1969 .

[32]  Avimanyu Sahoo,et al.  Neural network-based adaptive event-triggered control of nonlinear continuous-time systems , 2013, 2013 IEEE International Symposium on Intelligent Control (ISIC).

[33]  H. Witsenhausen.  A Counterexample in Stochastic Optimum Control , 1968 .