Output-Feedback Control of Unknown Linear Discrete-Time Systems With Stochastic Measurement and Process Noise via Approximate Dynamic Programming

This paper studies the optimal output-feedback control problem for unknown linear discrete-time systems with stochastic measurement and process noise. A dithered Bellman equation with the innovation covariance matrix is constructed via the expectation operator given in the form of a finite summation. On this basis, an output-feedback-based approximate dynamic programming method is developed, where the terms depending on the innovation covariance matrix are available with the aid of the innovation covariance matrix identified beforehand. Therefore, by iterating the Bellman equation, the resulting value function can converge to the optimal one in the presence of the aforementioned noise, and the nearly optimal control laws are delivered. To show the effectiveness and the advantages of the proposed approach, a simulation example and a velocity control experiment on a dc machine are employed.

[1]  Guang-Hong Yang,et al.  Data-driven output-feedback fault-tolerant control for unknown dynamic systems with faults changing system dynamics , 2016 .

[2]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[3]  Manfred Morari,et al.  Linear controller design for chance constrained systems , 2014, Autom..

[4]  E. Jury,et al.  ALMOST SURE BOUNDEDNESS OF RANDOMLY SAMPLED SYSTEMS , 1971 .

[5]  Steven X. Ding,et al.  Data-driven design of monitoring and diagnosis systems for dynamic processes: A review of subspace technique based schemes and some recent results , 2014 .

[6]  Gene H. Golub,et al.  Matrix computations , 1983 .

[7]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[8]  Shaocheng Tong,et al.  Observer-Based Adaptive Fuzzy Decentralized Optimal Control Design for Strict-Feedback Nonlinear Large-Scale Systems , 2018, IEEE Transactions on Fuzzy Systems.

[9]  Vasile Dragan,et al.  Optimal Stationary Dynamic Output-Feedback Controllers for Discrete-Time Linear Systems With Markovian Jumping Parameters and Additive White Noise Perturbations , 2016, IEEE Transactions on Automatic Control.

[10]  Yongduan Song,et al.  A novel approach to output feedback control of fuzzy stochastic systems , 2014, Autom..

[11]  U. Kruger,et al.  Dynamic Principal Component Analysis Using Subspace Model Identification , 2005, ICIC.

[12]  Lennart Ljung,et al.  A novel subspace identification approach with enforced causal models , 2005, Autom..

[13]  Shaocheng Tong,et al.  Observed-Based Adaptive Fuzzy Decentralized Tracking Control for Switched Uncertain Nonlinear Large-Scale Systems With Dead Zones , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[14]  Frank L. Lewis,et al.  Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics , 2014, Autom..

[15]  Daniel Kuhn,et al.  Distributionally Robust Control of Constrained Stochastic Systems , 2016, IEEE Transactions on Automatic Control.

[16]  R. Mehra.  On the identification of variances and adaptive Kalman filtering , 1970 .

[17]  Bin Yang,et al.  Rotation-based RLS algorithms: unified derivations, numerical properties, and parallel implementations , 1992, IEEE Trans. Signal Process..

[18]  Michael Athans,et al.  The Matrix Minimum Principle , 1967, Inf. Control..

[19]  Baocang Ding,et al.  On dynamic output feedback robust MPC for constrained quasi-LPV systems , 2013, Int. J. Control.

[20]  Tony Gustafsson,et al.  Instrumental variable subspace tracking using projection approximation , 1998, IEEE Trans. Signal Process..

[21]  Frank L. Lewis,et al.  Reinforcement Learning and Approximate Dynamic Programming for Feedback Control , 2012 .

[22]  Lei Guo,et al.  Observer-Based Feedback Controller Design for a Class of Stochastic Systems With Non-Gaussian Variables , 2015, IEEE Transactions on Automatic Control.

[23]  Frank L. Lewis,et al.  Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data , 2015, IEEE Transactions on Cybernetics.

[24]  Frank L. Lewis,et al.  Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control , 2007, Autom..

[25]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Si-Zhao Joe Qin,et al.  An overview of subspace identification , 2006, Comput. Chem. Eng..

[27]  Yong Zhang,et al.  Data-driven realizations of kernel and image representations and their application to fault detection and control system design , 2014, Autom..

[28]  Jin Wang,et al.  Closed-loop subspace identification using the parity space , 2006, Autom..

[29]  John Lygeros,et al.  Stochastic receding horizon control with output feedback and bounded controls , 2012, Autom..

[30]  Acpm Ton Backx,et al.  Generalized predictive control tuning by controller matching , 2015 .

[31]  Frank L. Lewis,et al.  Optimal control of nonlinear discrete time-varying systems using a new neural network approximation structure , 2015, Neurocomputing.

[32]  Guang-Hong Yang,et al.  Data-Driven Output-Feedback Fault-Tolerant Compensation Control for Digital PID Control Systems With Unknown Dynamics , 2016, IEEE Transactions on Industrial Electronics.

[33]  Marcello Farina,et al.  An approach to output-feedback MPC of stochastic linear discrete-time systems , 2015, Autom..

[34]  T. Kailath,et al.  An innovations approach to least-squares estimation--Part II: Linear smoothing in additive white noise , 1968 .

[35]  Guang-Hong Yang,et al.  New Results on Output Feedback $H_{\infty} $ Control for Linear Discrete-Time Systems , 2014, IEEE Transactions on Automatic Control.

[36]  Michel Verhaegen,et al.  Recursive subspace identification of linear and non-linear Wiener state-space models , 2000, Autom..

[37]  Torben Knudsen.  Consistency analysis of subspace identification methods based on a linear regression approach , 2001, Autom..

[38]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[39]  H. Akaike,et al.  Comment on "An innovations approach to least-squares estimation, part I: Linear filtering in additive white noise" , 1970 .

[40]  Ju H. Park,et al.  Robust static output feedback H∞ control design for linear systems with polytopic uncertainties , 2015, Syst. Control. Lett..

[41]  Guang-Hong Yang,et al.  Data-driven output-feedback fault-tolerant L2 control of unknown dynamic systems. , 2016, ISA transactions.

[42]  Andrew G. Barto,et al.  Adaptive linear quadratic control using policy iteration , 1994, Proceedings of 1994 American Control Conference - ACC '94.