Stochastic optimal control design for nonlinear networked control system via neuro dynamic programming using input-output measurements

Neuro dynamic programming (NDP) techniques for optimal control of nonlinear network control system (NNCS) are not addressed in the literature. Therefore, in this paper, a novel NNCS representation incorporating the unknown system uncertainties and network imperfections is introduced first by using input and output measurements. Then, an online neural network (NN) identifier is introduced to estimate the control coefficient matrix. Subsequently, the critic NN and action NN are employed along with the NN identifier to determine the forward-in-time, time-based stochastic optimal control of NNCS without using value and policy iterations. Instead, value function and control inputs are updated at every sampling instant. Lyapunov theory is used to show that all the closed-loop signals and NN weights are uniformly ultimately bounded (UUB) while the approximated control input converges close to its target value with time.

[1]  Asok Ray,et al.  A Stochastic Regulator for Integrated Communication and Control Systems: Part I—Formulation of Control Law , 1991 .

[2]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Yuanqing Xia,et al.  Design and Stability Criteria of Networked Predictive Control Systems With Random Network Delay in the Feedback Channel , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Minh Q. Phan,et al.  Identification of a Multistep-Ahead Observer and Its Application to Predictive Control , 1997 .

[5]  Sarangapani Jagannathan,et al.  Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[6]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[7]  Paul J. Webros.  A menu of designs for reinforcement learning over time , 1990 .

[8]  Wei Zhang,et al.  Stability of networked control systems , 2001 .

[9]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Radoslaw Romuald Zakrzewski,et al.  Neural network control of nonlinear discrete time systems , 1994 .

[11]  Johan Nilsson,et al.  Stochastic Analysis and Control of Real-Time Systems with Random Time Delays , 1996 .

[12]  O. Beldiman,et al.  Asymptotic behavior of networked control systems , 1999, Proceedings of the 1999 IEEE International Conference on Control Applications (Cat. No.99CH36328).

[13]  Frank L. Lewis,et al.  Optimal Control , 1986 .