Event-triggered constrained control with DHP implementation for nonaffine discrete-time systems

Abstract This paper proposes an event-based near-optimal control algorithm for nonaffine discrete-time systems with constrained inputs. The method is derived from the dual heuristic dynamic programming (DHP) technique. The challenge caused by saturating actuators is overcome by using a nonquadratic performance index. Then, the event-based control technique is used to decrease the amount of computation. Meanwhile, the stability analysis is provided. It illustrates that the proposed event-based method can asymptotically stabilize the nonaffine systems by using the Lyapunov method. Furthermore, the stability conditions and the design process of the event-based controller are established. The event-based DHP algorithm is implemented by constructing three neural networks, namely, the model network, the critic network, and the action network. Finally, simulation studies are conducted to demonstrate the applicability and the performance of the proposed method.

[1]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[2]  D. Bernstein.  Optimal nonlinear, but continuous, feedback control of systems with saturating actuators , 1995 .

[3]  Jing Na,et al.  Observer-based adaptive optimal control for unknown singularly perturbed nonlinear systems with input constraints , 2017, IEEE/CAA Journal of Automatica Sinica.

[4]  Dongbin Zhao,et al.  Comprehensive comparison of online ADP algorithms for continuous-time optimal control , 2017, Artificial Intelligence Review.

[5]  Eduardo Sontag,et al.  A general result on the stabilization of linear systems using bounded controls , 1994, IEEE Trans. Autom. Control..

[6]  Qichao Zhang,et al.  Event-Triggered $H_\infty $ Control for Continuous-Time Nonlinear System via Concurrent Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[7]  Haibo He,et al.  Model-Free Dual Heuristic Dynamic Programming , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Richard S. Sutton,et al.  A Menu of Designs for Reinforcement Learning Over Time , 1995 .

[9]  Indra Narayan Kar,et al.  Suboptimal robust stabilization of discrete-time mismatched nonlinear system , 2018, IEEE/CAA Journal of Automatica Sinica.

[10]  Zhong-Ping Jiang,et al.  Input-to-state stability for discrete-time nonlinear systems , 1999 .

[11]  Dimos V. Dimarogonas,et al.  Event-triggered control for discrete-time systems , 2010, Proceedings of the 2010 American Control Conference.

[12]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[13]  Hao Xu,et al.  Finite-Horizon Near-Optimal Output Feedback Neural Network Control of Quantized Nonlinear Discrete-Time Systems With Input Constraint , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Yang Xiong,et al.  Adaptive Dynamic Programming with Applications in Optimal Control , 2017 .

[15]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[16]  Haibo He,et al.  Event-triggered reinforcement learning approach for unknown nonlinear continuous-time system , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[17]  Derong Liu,et al.  Data-based robust optimal control of continuous-time affine nonlinear systems with matched uncertainties , 2016, Inf. Sci..

[18]  Ying Li,et al.  Adaptive dynamic programming for security of networked control systems with actuator saturation , 2018, Inf. Sci..

[19]  Frank L. Lewis,et al.  Off-Policy Reinforcement Learning for Synchronization in Multiagent Graphical Games , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Andrew R. Teel,et al.  Control of linear systems with saturating actuators , 1996 .

[21]  Chaoxu Mu,et al.  Developing nonlinear adaptive optimal regulators through an improved neural learning mechanism , 2016, Science China Information Sciences.

[22]  Haibo He,et al.  Goal Representation Heuristic Dynamic Programming on Maze Navigation , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Bernard Widrow,et al.  Punish/Reward: Learning with a Critic in Adaptive Threshold Systems , 1973, IEEE Trans. Syst. Man Cybern..

[24]  Huaguang Zhang,et al.  Iterative ADP learning algorithms for discrete-time multi-player games , 2018, Artificial Intelligence Review.

[25]  Haibo He,et al.  Event-Driven Nonlinear Discounted Optimal Regulation Involving a Power System Application , 2017, IEEE Transactions on Industrial Electronics.

[26]  Haibo He,et al.  Adaptive Event-Triggered Control Based on Heuristic Dynamic Programming for Nonlinear Discrete-Time Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[27]  S. Lyshevski.  Nonlinear discrete-time systems: constrained optimization and application of nonquadratic costs , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[28]  Derong Liu,et al.  Learning and Guaranteed Cost Control With Event-Based Adaptive Critic Implementation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Derong Liu,et al.  Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming , 2018, IEEE/CAA Journal of Automatica Sinica.

[30]  Haibo He,et al.  Adaptive Critic Nonlinear Robust Control: A Survey , 2017, IEEE Transactions on Cybernetics.

[31]  Haibo He,et al.  Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints , 2017, IEEE Trans. Neural Networks Learn. Syst..

[32]  Wei He,et al.  Iterative Learning Control of a Robotic Arm Experiment Platform with Input Constraint , 2018, IEEE Transactions on Industrial Electronics.