Event-driven optimal control for uncertain nonlinear systems with external disturbance via adaptive dynamic programming

Abstract This paper investigates the optimal control of uncertain nonlinear systems with external disturbance via adaptive dynamic programming, where two kinds of controllers are introduced and event-driven strategy is used to update the controllers as well as the network weights. One controller is constrained-input, and another is used to offset the external disturbance. Three kinds of neural networks (identifier network, critic network and actor network) are used to approximate the uncertain dynamics, the optimal value, control inputs and external disturbance, respectively. To the best knowledge of the authors, the advantage of this paper is that almost all the existing systems about optimal control via adaptive dynamic programming can be considered as special cases of this system, where the external disturbance is considered. By designing appropriate parameters, the uncertain nonlinear system is asymptotically stable with the event-driven controller. And, under the given learning rates of neural networks, the system state and the estimation error of neural networks are proved to be uniform ultimate boundedness (UUB) via adaptive dynamic programming. Finally, numerical simulation results are presented to verify the theoretical analysis.

[1]  Haibo He,et al.  GrDHP: A General Utility Function Representation for Dual Heuristic Dynamic Programming , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Karel Jezernik,et al.  An Event-Driven Approach to the Current Control of a BLDC Motor Using FPGA , 2014, IEEE Transactions on Industrial Electronics.

[3]  Paulo Tabuada,et al.  Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks , 2007, IEEE Transactions on Automatic Control.

[4]  Frank L. Lewis,et al.  Nonlinear feedback control of a gantry crane , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[5]  K. Vamvoudakis.  Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[6]  Frank L. Lewis,et al.  Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems , 2014, Autom..

[7]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[8]  Dongbin Zhao,et al.  Iterative Adaptive Dynamic Programming for Solving Unknown Nonlinear Zero-Sum Game Based on Online Data , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Haibo He,et al.  Event-Triggered Optimal Control for Partially Unknown Constrained-Input Systems via Adaptive Dynamic Programming , 2017, IEEE Transactions on Industrial Electronics.

[10]  Ramazan Coban,et al.  Artificial Bee Colony Algorithm Based Linear Quadratic Optimal Controller Design for a Nonlinear Inverted Pendulum , 2015 .

[11]  Ramazan Coban.  A context layered locally recurrent neural network for dynamic system identification , 2013, Eng. Appl. Artif. Intell..

[12]  Derong Liu,et al.  Event-based input-constrained nonlinear H∞ state feedback with adaptive critic and neural implementation , 2016, Neurocomputing.

[13]  Xiaofeng Wang,et al.  On event design in event-triggered feedback systems , 2011, Autom..

[14]  Tongwen Chen,et al.  Event detection and control co-design of sampled-data systems , 2014, Int. J. Control.

[15]  Eloy García,et al.  Model-Based Event-Triggered Control for Systems With Quantization and Time-Varying Network Delays , 2013, IEEE Transactions on Automatic Control.

[16]  Tao Dong,et al.  Event-triggered synchronization strategy for complex dynamical networks with the Markovian switching topologies , 2016, Neural Networks.

[17]  Xiaofeng Liao,et al.  On the general consensus protocol in multi-agent networks with second-order dynamics and sampled data , 2016 .

[18]  B. Finlayson.  The method of weighted residuals and variational principles : with application in fluid mechanics, heat and mass transfer , 1972 .

[19]  Xiaofeng Liao,et al.  Impact of discontinuous antivirus strategy in a computer virus model with the point to group , 2016 .

[20]  Dongbin Zhao,et al.  Self-teaching adaptive dynamic programming for Gomoku , 2012, Neurocomputing.

[21]  Yu Jiang,et al.  Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Frank L. Lewis,et al.  Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control , 2007, Autom..

[23]  Huaguang Zhang,et al.  Optimal tracking control for completely unknown nonlinear discrete-time Markov jump systems using data-based reinforcement learning method , 2016, Neurocomputing.

[24]  Avimanyu Sahoo,et al.  Adaptive Neural Network-Based Event-Triggered Control of Single-Input Single-Output Nonlinear Discrete-Time Systems , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Bin Jiang,et al.  Online Adaptive Policy Learning Algorithm for $H_{\infty }$ State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems , 2014, IEEE Transactions on Cybernetics.

[26]  Tongwen Chen,et al.  Event based agreement protocols for multi-agent networks , 2013, Autom..

[27]  Qichao Zhang,et al.  Event-Triggered $H_\infty $ Control for Continuous-Time Nonlinear System via Concurrent Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[28]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[29]  Tao Dong,et al.  Event‐Triggered Consensus of Nonlinear Multi‐Agent Systems with Unknown External Disturbance , 2018 .

[30]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[31]  Ramazan Coban.  Dynamical adaptive integral backstepping variable structure controller design for uncertain systems and experimental application , 2017 .

[32]  Derong Liu,et al.  Neural-network-based zero-sum game for discrete-time nonlinear systems via iterative adaptive dynamic programming algorithm , 2013, Neurocomputing.

[33]  Avimanyu Sahoo,et al.  Near Optimal Event-Triggered Control of Nonlinear Discrete-Time Systems Using Neurodynamic Programming , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Qichao Zhang,et al.  Data-driven adaptive dynamic programming for continuous-time fully cooperative games with partially constrained inputs , 2017, Neurocomputing.

[35]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[36]  Junqiang Xi,et al.  Special Issue on “Recent Developments on Modeling and Control of Hybrid Electric Vehicles” , 2016 .

[37]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[38]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[39]  Haibo He,et al.  Event-triggered adaptive dynamic programming for continuous-time nonlinear system using measured input-output data , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).