Decentralized Event-Triggered Adaptive Control of Discrete-Time Nonzero-Sum Games Over Wireless Sensor-Actuator Networks With Input Constraints

This article studies an event-triggered communication and adaptive dynamic programming (ADP) co-design control method for the multiplayer nonzero-sum (NZS) games of a class of nonlinear discrete-time wireless sensor-actuator network (WSAN) systems subject to input constraints. By virtue of the ADP algorithm, the critic and actor networks are established to attain the approximate Nash equilibrium point solution in the context of the constrained control mechanism. Simultaneously, as the sensors and actuators are physically distributed, a decentralized event-triggered communication protocol is presented, accompanied by a dead-zone operation which avoids the unnecessary events. By predefining the triggering thresholds and compensation values, a novel adaptive triggering condition is derived to guarantee the stability of the event-based closed-loop control system. Then resorting to the Lyapunov theory, the system states and the critic/actor network weight estimation errors are proven to be ultimately bounded. Moreover, an explicit analysis on the nontriviality of the interevent times is also provided. Finally, two numerical examples are conducted to validate the effectiveness of the proposed method.

[1]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Wei Xing Zheng,et al.  Dynamic Event-Based Control of Nonlinear Stochastic Systems , 2017, IEEE Transactions on Automatic Control.

[3]  Avimanyu Sahoo,et al.  Near Optimal Event-Triggered Control of Nonlinear Discrete-Time Systems Using Neurodynamic Programming , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Frank L. Lewis,et al.  Applied optimal control & estimation : digital design & implementation , 1992 .

[5]  Lihua Xie,et al.  Asynchronous periodic event-triggered consensus for multi-agent systems , 2017, Autom..

[6]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[7]  Wei Wu,et al.  Event‐triggered fault‐tolerant control and scheduling codesign for nonlinear networked control systems with medium‐access constraint and packet disordering , 2018 .

[8]  Xiangwei Bu,et al.  Actor-Critic Reinforcement Learning Control of Non-Strict Feedback Nonaffine Dynamic Systems , 2019, IEEE Access.

[9]  K. Vamvoudakis Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[10]  Panganamala Ramana Kumar,et al.  Optimizing controller location in networked control systems with packet drops , 2008, IEEE Journal on Selected Areas in Communications.

[11]  Chaomin Luo,et al.  Discrete-Time Nonzero-Sum Games for Multiplayer Using Policy-Iteration-Based Adaptive Dynamic Programming Algorithms , 2017, IEEE Transactions on Cybernetics.

[12]  Huaguang Zhang,et al.  Event-trigger-based robust control for nonlinear constrained-input systems using reinforcement learning method , 2019, Neurocomputing.

[13]  Huaguang Zhang,et al.  Neural network-based online H∞ control for discrete-time affine nonlinear system using adaptive dynamic programming , 2016, Neurocomputing.

[14]  Yixin Chen,et al.  Real-Time Wireless Sensor-Actuator Networks for Industrial Cyber-Physical Systems , 2016, Proceedings of the IEEE.

[15]  Kyriakos G. Vamvoudakis,et al.  Non-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems , 2015, Autom..

[16]  Nader Meskin,et al.  Event-Triggered Suboptimal Tracking Controller Design for a Class of Nonlinear Discrete-Time Systems , 2017, IEEE Transactions on Industrial Electronics.

[17]  Kun Zhang,et al.  Data-driven adaptive dynamic programming schemes for non-zero-sum games of unknown discrete-time nonlinear systems , 2018, Neurocomputing.

[18]  Mohammad-Bagher Naghibi-Sistani,et al.  Reinforcement learning and neural networks for multi-agent nonzero-sum games of nonlinear constrained-input systems , 2014, International Journal of Machine Learning and Cybernetics.

[19]  Derong Liu,et al.  On Mixed Data and Event Driven Design for Adaptive-Critic-Based Nonlinear $H_{\infty}$ Control , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Dong Yue,et al.  Relaxed Real-Time Scheduling Stabilization of Discrete-Time Takagi–Sugeno Fuzzy Systems via An Alterable-Weights-Based Ranking Switching Mechanism , 2018, IEEE Transactions on Fuzzy Systems.

[21]  Frank L. Lewis,et al.  Actor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[23]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[24]  Manuel Mazo,et al.  Decentralized Event-Triggered Control Over Wireless Sensor/Actuator Networks , 2010, IEEE Transactions on Automatic Control.

[25]  Manuel Mazo,et al.  System Architectures, Protocols and Algorithms for Aperiodic Wireless Control Systems , 2014, IEEE Transactions on Industrial Informatics.

[26]  Qichao Zhang,et al.  Event-Triggered $H_\infty $ Control for Continuous-Time Nonlinear System via Concurrent Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[27]  Bowen Zhou,et al.  Fault Diagnosis for Energy Internet Using Correlation Processing-Based Convolutional Neural Networks , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[28]  Sandra Hirche,et al.  Event-Triggered State Estimation: An Iterative Algorithm and Optimality Properties , 2017, IEEE Transactions on Automatic Control.

[29]  Avimanyu Sahoo,et al.  Neural Network-Based Event-Triggered State Feedback Control of Nonlinear Continuous-Time Systems , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Derong Liu,et al.  Policy Iteration Adaptive Dynamic Programming Algorithm for Discrete-Time Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Xiangpeng Xie,et al.  Event-Triggered Integral Sliding-Mode Control for Nonlinear Constrained-Input Systems With Disturbances via Adaptive Dynamic Programming , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[32]  Qichao Zhang,et al.  Experience Replay for Optimal Control of Nonzero-Sum Game Systems With Unknown Dynamics , 2016, IEEE Transactions on Cybernetics.

[33]  Frank L. Lewis,et al.  Online solution of nonlinear two-player zero-sum games using synchronous policy iteration , 2010, 49th IEEE Conference on Decision and Control (CDC).

[34]  P. Olver Nonlinear Systems , 2013 .

[35]  Qing-Long Han,et al.  On asynchronous event-triggered control of decentralized networked systems , 2018, Inf. Sci..

[36]  Paulo Tabuada,et al.  Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks , 2007, IEEE Transactions on Automatic Control.

[37]  Dong Liu,et al.  Event-based model-free adaptive control for discrete-time non-linear processes , 2017 .

[38]  Kyriakos G. Vamvoudakis,et al.  Asymptotically Stable Adaptive–Optimal Control Algorithm With Saturating Actuators and Relaxed Persistence of Excitation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Qichao Zhang,et al.  Event-Triggered H ∞ Control for Continuous-Time Nonlinear System , 2015, ISNN.

[40]  Sebastian Trimpe,et al.  Distributed Event-Based State Estimation for Networked Systems: An LMI Approach , 2017, IEEE Transactions on Automatic Control.

[41]  Meng Joo Er,et al.  Event-Triggered Consensus of Linear Multiagent Systems With Time-Varying Communication Delays , 2020, IEEE Transactions on Cybernetics.

[42]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.