Stochastic Inverse optimal control of unknown linear networked control system in the presence of random delays and packet losses

In this paper, the inverse optimal control approach is applied to stabilization in probability of unknown linear networked control system (NCS) in presence of random delays and packet losses. The proposed control scheme is based on Kalman filter parameter estimation to solve the infinite horizon regulator problem for NCS with stochastic system matrices, and avoids to solve the associated stochastic Riccati equation (SRE); additionally a cost functional is minimized. The stabilizing optimal controller is based on a discrete-time stochastic control Lyapunov function.

[1]  K. Åström.  Introduction to Stochastic Control Theory , 1970 .

[2]  Miroslav Krstic,et al.  Stabilization of Nonlinear Uncertain Systems , 1998 .

[3]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[4]  Sarangapani Jagannathan,et al.  Adaptive dynamic programming-based optimal control of unknown affine nonlinear discrete-time systems , 2009, 2009 International Joint Conference on Neural Networks.

[5]  Edgar N. Sanchez,et al.  Discrete-Time Inverse Optimal Control for Nonlinear Systems , 2013 .

[6]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[7]  B. Anderson,et al.  Optimal control: linear quadratic methods , 1990 .

[8]  Bruno Sinopoli,et al.  Foundations of Control and Estimation Over Lossy Networks , 2007, Proceedings of the IEEE.

[9]  Abbas Jamalipour,et al.  Wireless communications , 2005, GLOBECOM '05. IEEE Global Telecommunications Conference, 2005..

[10]  Johan Nilsson,et al.  Stochastic Analysis and Control of Real-Time Systems with Random Time Delays , 1996 .

[11]  Qixin Zhu,et al.  Stochastic optimal control and analysis of stability of networked control systems with long delay , 2003, Autom..

[12]  Mrdjan J. Jankovic,et al.  Constructive Nonlinear Control , 2011 .

[13]  Bart De Schutter,et al.  Reinforcement Learning and Dynamic Programming Using Function Approximators , 2010 .

[14]  Asok Ray,et al.  Integrated Communication and Control Systems: Part I—Analysis , 1988 .

[15]  Karl Johan Åström,et al.  Adaptive Control , 1989, Embedded Digital Control with Microcontrollers.

[16]  Lei Guo.  Estimating time-varying parameters by the Kalman filter based algorithm: stability and convergence , 1990 .

[17]  Donald E. Kirk,et al.  Optimal control theory : an introduction , 1970 .

[18]  B. Azimi-Sadjadi,et al.  Stability of networked control systems in the presence of packet losses , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[19]  Kevin Fiedler,et al.  Robust Nonlinear Control Design State Space And Lyapunov Techniques , 2016 .

[20]  Linda Bushnell,et al.  Stability analysis of networked control systems , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[21]  Paul J. Webros.  A menu of designs for reinforcement learning over time , 1990 .

[22]  Ying Lin,et al.  Particle Swarm Optimization With an Aging Leader and Challengers , 2013, IEEE Transactions on Evolutionary Computation.

[23]  Wei Zhang,et al.  Stability of networked control systems , 2001 .

[24]  Wei Zhang,et al.  Stability of networked control systems: explicit analysis of delay , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[25]  Jing Wu,et al.  Design of Networked Control Systems With Packet Dropouts , 2007, IEEE Transactions on Automatic Control.

[26]  R. Bucy,et al.  Filtering for stochastic processes with applications to guidance , 1968 .

[27]  Edgar N. Sánchez,et al.  Inverse optimal control for discrete-time stochastic nonlinear systems stabilization , 2013, 2013 American Control Conference.

[28]  R. E. Kalman,et al.  When Is a Linear Control System Optimal , 1964 .

[29]  R.A. Freeman,et al.  Optimal nonlinear controllers for feedback linearizable systems , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[30]  J. Willems,et al.  Inverse optimal control problem for linear discrete-time systems , 1977 .

[31]  John B. Moore,et al.  Persistence of Excitation in Linear Systems , 1985, 1985 American Control Conference.

[32]  Nathan van de Wouw,et al.  Stability of Networked Control Systems With Uncertain Time-Varying Delays , 2009, IEEE Transactions on Automatic Control.

[33]  Hu Shousong,et al.  Brief Stochastic optimal control and analysis of stability of networked control systems with long delay , 2003 .

[34]  Antonio Barreiro,et al.  Analysis of networked control systems with drops and variable delays , 2007, Autom..

[35]  Randy A. Freeman,et al.  Robust Nonlinear Control Design , 1996 .

[36]  Hao Xu,et al.  Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses , 2012, Autom..

[37]  Asok Ray,et al.  A Stochastic Regulator for Integrated Communication and Control Systems: Part I—Formulation of Control Law , 1991 .

[38]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.