Infinite-Horizon Linear-Quadratic-Gaussian Control with Costly Measurements

In this paper, we consider an infinite horizon Linear-Quadratic-Gaussian control problem with controlled and costly measurements. A control strategy and a measurement strategy are co-designed to optimize the trade-off among control performance, actuating costs, and measurement costs. We address the co-design and co-optimization problem by establishing a dynamic programming equation with controlled lookahead. By leveraging the dynamic programming equation, we fully characterize the optimal control strategy and the measurement strategy analytically. The optimal control is linear in the state estimate that depends on the measurement strategy. We prove that the optimal measurement strategy is independent of the measured state and is periodic. And the optimal period length is determined by the cost of measurements and system parameters. We demonstrate the potential application of the co-design and co-optimization problem in an optimal self-triggered control paradigm. Two examples are provided to show the effectiveness of the optimal measurement strategy in reducing the overhead of measurements while keeping the system performance.

[1]  Richard H. Middleton,et al.  Networked control design for linear systems , 2003, Autom..

[2]  Sandra Hirche,et al.  On LQG joint optimal scheduling and control under communication constraints , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[3]  Quanyan Zhu,et al.  Cross-Layer Coordinated Attacks on Cyber-Physical Systems: A LQG Game Framework with Controlled Observations , 2020, ArXiv.

[4]  M. Athans.  On the Determination of Optimal Costly Measurement Strategies for Linear Stochastic Systems , 1972 .

[5]  T. Başar,et al.  Optimal Estimation with Limited Measurements , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[6]  John S. Baras,et al.  Linear quadratic games with costly measurements , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[7]  Richard W. Longman,et al.  Optimal selection of observation times in the linear-quadratic Gaussian control problem , 1983 .

[8]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[9]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[10]  J.S. Baras,et al.  Multiple Sampling for Estimation on a Finite Horizon , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[11]  L. Schulman,et al.  On LQG control with communication power constraint , 2007, 2007 European Control Conference (ECC).

[12]  R. Durrett.  Probability: Theory and Examples , 1993 .

[13]  Wei Wu,et al.  Optimal Sensor Querying: General Markovian and LQG Models With Controlled Observations , 2008, IEEE Transactions on Automatic Control.

[14]  Paulo Tabuada,et al.  Self-triggered linear quadratic control , 2014, Autom..

[15]  C. A. Cooper,et al.  An optimal stochastic control problem with observation cost , 1971 .

[17]  D. Sworder,et al.  Introduction to stochastic control , 1972 .

[18]  J. Peschon,et al.  Optimal control of measurement subsystems , 1967, IEEE Transactions on Automatic Control.

[19]  E. Kreyszig.  Introductory Functional Analysis With Applications , 1978 .

[20]  George J. Pappas,et al.  LQG Control and Sensing Co-Design , 2018, IEEE Transactions on Automatic Control.

[21]  Ufuk Topcu,et al.  Stochastic Games with Sensing Costs , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[22]  Quanyan Zhu,et al.  Continuous-Time Markov Decision Processes with Controlled Observations , 2019, 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  Harold J. Kushner,et al.  On the optimum timing of observations for linear control systems with unknown initial state , 1964 .

[24]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[25]  Tamer Basar,et al.  Optimal communication scheduling and remote estimation over an additive noise channel , 2016, Autom..

[26]  F. Hutter,et al.  Towards TempoRL: Learning When to Act , 2020 .

[27]  Hideaki Ishii,et al.  Self-triggered control with tradeoffs in communication and computation , 2018, Autom..