Event-triggered reinforcement learning H∞ control design for constrained-input nonlinear systems subject to actuator failures

Abstract In the paper, a novel input-constrained H ∞ fault-tolerant control approach is developed by using sliding mode control technology and event-triggered reinforcement learning (RL) algorithm. To reduce or even eliminate the impacts of the time-varying actuator failures, a properly sliding mode control strategy is proposed for the controlled system, while the event-triggered H ∞ control scheme is established via RL algorithm for the equivalent sliding mode dynamics. By utilizing a single neural network (NN), the Hamilton–Jacobi–Bellman (HJB) equation can be solved approximately, thereby gaining time-triggered worst-case disturbance law, as well as event-triggered optimal control policy. Besides, it is unnecessary to given a initial stabilizing control input in the learning process of neural networks (NNs) in this paper. Moreover, the Lyapunov stability principle is applied to guarantee that the controlled system is uniformly ultimately bounded (UUB). Finally, to verify the feasibility and efficient performance of the developed approach, three simulations are carried out.

[1]  Bin Jiang,et al.  Online Adaptive Policy Learning Algorithm for $H_{\infty }$ State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems , 2014, IEEE Transactions on Cybernetics.

[2]  Qingling Zhang,et al.  Finite-time synchronization for second-order nonlinear multi-agent system via pinning exponent sliding mode control. , 2016, ISA transactions.

[3]  Cristina Alcaraz,et al.  WASAM: A dynamic wide-area situational awareness model for critical domains in Smart Grids , 2014, Future Gener. Comput. Syst..

[4]  Marios M. Polycarpou,et al.  A Coordinated Communication Scheme for Distributed Fault Tolerant Control , 2013, IEEE Transactions on Industrial Informatics.

[5]  Gang Tao,et al.  Adaptive Fault-Tolerant Control of Uncertain Nonlinear Large-Scale Systems With Unknown Dead Zone , 2016, IEEE Transactions on Cybernetics.

[6]  Ming Wang,et al.  Situation-Aware Dynamic Service Coordination in an IoT Environment , 2017, IEEE/ACM Transactions on Networking.

[7]  Heng Nian,et al.  Simplified Modulation Scheme for Open-End Winding PMSM System With Common DC Bus Under Open-Phase Fault Based on Circulating Current Suppression , 2020, IEEE Transactions on Power Electronics.

[8]  Huaguang Zhang,et al.  Optimal Guaranteed Cost Sliding Mode Control for Constrained-Input Nonlinear Systems With Matched and Unmatched Disturbances , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Marios M. Polycarpou,et al.  Decentralized Fault Tolerant Control of a Class of Interconnected Nonlinear Systems , 2011, IEEE Transactions on Automatic Control.

[10]  Huaguang Zhang,et al.  Neural network-based online H∞ control for discrete-time affine nonlinear system using adaptive dynamic programming , 2016, Neurocomputing.

[11]  Thomas Parisini,et al.  Plug-and-Play Fault Detection and Control-Reconfiguration for a Class of Nonlinear Large-Scale Constrained Systems , 2014, IEEE Transactions on Automatic Control.

[12]  Derong Liu,et al.  Event-based input-constrained nonlinear H∞ state feedback with adaptive critic and neural implementation , 2016, Neurocomputing.

[13]  Derong Liu,et al.  On Mixed Data and Event Driven Design for Adaptive-Critic-Based Nonlinear $H_{\infty}$ Control , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Songwu Lu,et al.  Robust nonlinear system identification using neural-network models , 1998, IEEE Trans. Neural Networks.

[15]  Yang Liu,et al.  Neural network-based H∞ sliding mode control for nonlinear systems with actuator faults and unmatched disturbances , 2018, Neurocomputing.

[16]  Marios M. Polycarpou,et al.  Learning methodology for failure detection and accommodation , 1995 .

[17]  Shijie Zhang,et al.  Sliding mode control of quantized systems against bounded disturbances , 2014, Inf. Sci..

[18]  Derong Liu,et al.  Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties , 2016, Inf. Sci..

[19]  Haibo He,et al.  An Event-Triggered ADP Control Approach for Continuous-Time System With Unknown Internal States , 2017, IEEE Transactions on Cybernetics.

[20]  Haibo He,et al.  Improving the Critic Learning for Event-Based Nonlinear $H_{\infty }$ Control Design , 2017, IEEE Transactions on Cybernetics.

[21]  Kun Zhang,et al.  Tracking control optimization scheme of continuous-time nonlinear system via online single network adaptive critic design method , 2017, Neurocomputing.

[22]  V. Utkin,et al.  Integral sliding mode in systems operating under uncertainty conditions , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[23]  Yi Shen,et al.  Adaptive fault-tolerant robust control for a linear system with adaptive fault identification , 2013 .

[24]  Guang-Hong Yang,et al.  Adaptive fault-tolerant control for affine non-linear systems based on approximate dynamic programming , 2016 .

[25]  Huaguang Zhang,et al.  General value iteration based single network approach for constrained optimal controller design of partially-unknown continuous-time nonlinear systems , 2018, J. Frankl. Inst..

[26]  Shaocheng Tong,et al.  Adaptive fuzzy decentralized tracking fault-tolerant control for stochastic nonlinear large-scale systems with unmodeled dynamics , 2014, Inf. Sci..

[27]  Vadim I. Utkin,et al.  Sliding mode control in electromechanical systems , 1999 .

[28]  Guang-Hong Yang,et al.  Robust adaptive fault‐tolerant control of uncertain linear systems via sliding‐mode output feedback , 2015 .

[29]  Huaguang Zhang,et al.  Adaptive Fault-Tolerant Tracking Control for MIMO Discrete-Time Systems via Reinforcement Learning Algorithm With Less Learning Parameters , 2017, IEEE Transactions on Automation Science and Engineering.

[30]  Guang-Hong Yang,et al.  Robust Adaptive Fault-Tolerant Control for a Class of Uncertain Nonlinear Time Delay Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Halim Alwi,et al.  Fault tolerant longitudinal aircraft control using non-linear integral sliding mode , 2014 .

[32]  Wen-Jun Cao,et al.  Nonlinear integral-type sliding surface for both matched and unmatched uncertain systems , 2004, IEEE Trans. Autom. Control..

[33]  Guang-Hong Yang,et al.  Asynchronous fault detection and robust control for switched systems with state reset strategy , 2018, J. Frankl. Inst..

[34]  Hamid Reza Karimi,et al.  $H_{\infty}$ Output Tracking Control for Networked Systems With Adaptively Adjusted Event-Triggered Scheme , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[35]  Wei Xing Zheng,et al.  Fault-Tolerant Cooperative Tracking Control via Integral Sliding Mode Control Technique , 2017, IEEE/ASME Transactions on Mechatronics.

[36]  Derong Liu,et al.  Learning and Guaranteed Cost Control With Event-Based Adaptive Critic Implementation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Jun Zhang,et al.  Adaptive Fault-Tolerant Tracking Control for Linear and Lipschitz Nonlinear Multi-Agent Systems , 2015, IEEE Transactions on Industrial Electronics.

[38]  Yongming Li,et al.  Observer-Based Adaptive Decentralized Fuzzy Fault-Tolerant Control of Nonlinear Large-Scale Systems With Actuator Failures , 2014, IEEE Transactions on Fuzzy Systems.

[39]  Qichao Zhang,et al.  Event-Triggered $H_\infty $ Control for Continuous-Time Nonlinear System via Concurrent Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[40]  Leila Parsa,et al.  Global Fault-Tolerant Control Technique for Multiphase Permanent-Magnet Machines , 2015, IEEE Transactions on Industry Applications.

[41]  Cristina Alcaraz,et al.  Analysis of requirements for critical control systems , 2012, Int. J. Crit. Infrastructure Prot..

[42]  Peng Shi,et al.  Adaptive Neural Fault-Tolerant Control of a 3-DOF Model Helicopter System , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[43]  Derong Liu,et al.  Neural-Network-Based Online HJB Solution for Optimal Robust Guaranteed Cost Control of Continuous-Time Uncertain Nonlinear Systems , 2014, IEEE Transactions on Cybernetics.

[44]  Xiong Yang,et al.  Adaptive Critic Designs for Event-Triggered Robust Control of Nonlinear Systems With Unknown Dynamics , 2019, IEEE Transactions on Cybernetics.