Model-Free Optimal Output Regulation for Linear Discrete-Time Lossy Networked Control Systems

In this article, a new model-free approach is proposed to solve the output regulation problem for networked control systems, where the system state can be lost in the feedback process. The goal of the output regulation is to design a control law that can make the system achieve asymptotic stability of the tracking error while maintaining the stability of the closed-loop system. The solvability of the output regulation problem depends on the solvability of a set of matrix equations called the regulator equations. First, a restructured dynamic system is established by using the Smith predictor; then, an off-policy algorithm based on reinforcement learning is developed to calculate the feedback gain using only the measured data when dropout occurs. Based on the solution to the feedback gain, a model-free solution is provided for solving the forward gain using the regulator equations. The simulation results demonstrate the effectiveness of the proposed approach for discrete-time networked systems with unknown dynamics and dropout.

[1]  Wei-Yong Yan,et al.  Stability robustness of networked control systems with respect to packet loss , 2007, Autom..

[2]  Wei Chen,et al.  Network-based H∞ output tracking control for systems with time-varying bounded delay , 2011, Proceedings 2011 International Conference on Transportation, Mechanical, and Electrical Engineering (TMEE).

[3]  Bruce A. Francis,et al.  The internal model principle of control theory , 1976, Autom..

[4]  Frank L. Lewis,et al.  Dual-Rate Operational Optimal Control for Flotation Industrial Process With Unknown Operational Model , 2019, IEEE Transactions on Industrial Electronics.

[5]  Frank L. Lewis,et al.  Data-Driven Flotation Industrial Process Operational Optimal Control Based on Reinforcement Learning , 2018, IEEE Transactions on Industrial Informatics.

[6]  E. Davison.  The robust control of a servomechanism problem for linear time-invariant multivariable systems , 1976 .

[7]  Frank L. Lewis,et al.  H∞ control of linear discrete-time systems: Off-policy reinforcement learning , 2017, Autom..

[8]  James Lam,et al.  A new delay system approach to network-based control , 2008, Autom..

[9]  Frank L. Lewis,et al.  Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control , 2007, Autom..

[10]  Huijun Gao,et al.  Network-Based ${{\cal H}}_{\!\!\!\infty }$ Output Tracking Control , 2008, IEEE Transactions on Automatic Control.

[11]  Wei Zhang,et al.  Stability of networked control systems , 2001 .

[12]  Bruno Sinopoli,et al.  Kalman filtering with intermittent observations , 2004, IEEE Transactions on Automatic Control.

[13]  Xuhui Bu,et al.  Model free adaptive control with data dropouts , 2011, Expert Syst. Appl..

[14]  C. Desoer,et al.  Linear Time-Invariant Robust Servomechanism Problem: A Self-Contained Exposition , 1980 .

[15]  Jie Huang,et al.  Robust output regulation problem for linear time-delay systems , 2015, Int. J. Control.

[16]  Mohamed Hamdy,et al.  Time-Varying Delay Compensation for a Class of Nonlinear Control Systems Over Network via $H_{\infty }$ Adaptive Fuzzy Controller , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  Derong Liu,et al.  Online Synchronous Approximate Optimal Learning Algorithm for Multi-Player Non-Zero-Sum Games With Unknown Dynamics , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[18]  Fuad E. Alsaadi,et al.  Output-Feedback Control for Nonlinear Stochastic Systems With Successive Packet Dropouts and Uniform Quantization Effects , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[19]  Frank L. Lewis,et al.  New Methods for Optimal Operational Control of Industrial Processes Using Reinforcement Learning on Two Time Scales , 2020, IEEE Transactions on Industrial Informatics.

[20]  Frank L. Lewis,et al.  Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics , 2014, Autom..

[21]  Frank L. Lewis,et al.  Operational Control of Mineral Grinding Processes Using Adaptive Dynamic Programming and Reference Governor , 2019, IEEE Transactions on Industrial Informatics.

[22]  Jie Huang,et al.  Nonlinear Output Regulation: Theory and Applications , 2004 .

[23]  Tianyou Chai,et al.  Setpoint dynamic compensation via output feedback control with network induced time delays , 2015, 2015 American Control Conference (ACC).

[24]  Frank L. Lewis,et al.  Off-Policy Interleaved $Q$ -Learning: Optimal Control for Affine Nonlinear Discrete-Time Systems , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Jie Huang,et al.  Output regulation problem for discrete-time linear time-delay systems by output feedback control , 2016 .

[26]  João Pedro Hespanha,et al.  A Survey of Recent Results in Networked Control Systems , 2007, Proceedings of the IEEE.

[27]  Frank L. Lewis,et al.  Tracking Control for Linear Discrete-Time Networked Control Systems With Unknown Dynamics and Dropout , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[28]  G. Hewer.  An iterative technique for the computation of the steady state gains for the discrete optimal regulator , 1971 .

[29]  Jianliang Wang,et al.  H∞ Controller Design of Networked Control Systems with Markov Packet Dropouts , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[30]  Bruno Sinopoli,et al.  Foundations of Control and Estimation Over Lossy Networks , 2007, Proceedings of the IEEE.

[31]  Tianyou Chai,et al.  Optimal Output Regulation of Linear Discrete-Time Systems With Unknown Dynamics Using Reinforcement Learning , 2020, IEEE Transactions on Cybernetics.

[32]  Zhong-Ping Jiang,et al.  Adaptive and optimal output feedback control of linear systems: An adaptive dynamic programming approach , 2014, Proceeding of the 11th World Congress on Intelligent Control and Automation.

[33]  P. Sannuti,et al.  On optimal output regulation for linear systems-state feedback , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[34]  Frank L. Lewis,et al.  Model-Free Linear Discrete-Time System H∞ Control Using Input-Output Data , 2018, 2018 International Conference on Advanced Mechatronic Systems (ICAMechS).

[35]  Zhong-Ping Jiang,et al.  Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[36]  Christopher I. Byrnes,et al.  Output regulation for linear distributed parameter systems , 2000, IEEE Trans. Autom. Control..

[37]  Zhong-Ping Jiang,et al.  Adaptive optimal output regulation via output-feedback: An adaptive dynamic programing approach , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[38]  B. Francis.  The linear multivariable regulator problem , 1976, 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes.

[39]  Tamer Basar,et al.  Optimal control of LTI systems over unreliable communication links , 2006, Autom..

[40]  Qingling Zhang,et al.  Integrated Sliding Mode Control and Neural Networks Based Packet Disordering Prediction for Nonlinear Networked Control Systems , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[41]  Jihong Wang,et al.  A New Method for Estimating the Maximum Allowable Delay in Networked Control of bounded nonlinear systems , 2011, The 17th International Conference on Automation and Computing.

[42]  Tianyou Chai,et al.  MPC-based setpoint compensation with unreliable wireless communications and constrained operational conditions , 2017, Neurocomputing.