A Linear-Quadratic Gaussian Approach to Dynamic Information Acquisition

We consider optimal information acquisition for the control of linear discrete-time random systems with noisy observations and apply the findings to the problem of dynamically implementing emissions-reduction targets. The optimal policy, which is provided in closed form, depends on a single composite parameter which determines the criticality of the system. For subcritical systems, it is optimal to perform “noise leveling,” that is, to reduce the variance of the state uncertainty to an optimal level and keep it constant by a steady feed of information updates. For critical systems, the optimal policy is “noise attenuation,” that is, to substantially decrease the variance once and never acquire information thereafter. Finally for supercritical systems, information acquisition is never in the best interest of the decision maker. In each case, an explicit expression of the value function is obtained. The criticality of the system, and therefore the tradeoff between spending resources on the control or on information to improve the control, is influenced by a “policy parameter” which determines the importance a decision maker places on uncertainty reduction. The dependence of the system performance on the policy parameter is illustrated using a practical climate-control problem where a regulator imposes state-contingent taxes to probabilistically attain emissions targets.

[1]  W. Rudin Principles of mathematical analysis , 1964 .

[2]  I. Lavalle On Cash Equivalents and Information Evaluation in Decisions Under Uncertainty Part I: Basic Theory , 1968 .

[3]  Lisa M. Maillart,et al.  Assessing Dynamic Breast Cancer Screening Policies , 2008, Oper. Res..

[4]  M. Cichon,et al.  Energy and Climate Change , 1997, Energy Exploration & Exploitation.

[5]  Michael Athans,et al.  1972 IFAC congress paper: On the determination of optimal costly measurement strategies for linear stochastic systems , 1972 .

[6]  Tamer Basar,et al.  Stochastic Networked Control Systems , 2013 .

[7]  W. T. Tucker Linear Estimation and Stochastic Control , 1984 .

[8]  Mark Salmon,et al.  Error Correction Mechanisms , 1982 .

[9]  H. Scarf THE OPTIMALITY OF (S,S) POLICIES IN THE DYNAMIC INVENTORY PROBLEM , 1959 .

[10]  Finn E. Kydland,et al.  Noncooperative and Dominant Player Solutions in Discrete Dynamic Games , 1975 .

[11]  Jean-Pascal van Ypersele de Strihou Climate Change 2014 - Synthesis Report , 2015 .

[12]  A. Dixit Optimization in Economic Theory , 1976 .

[13]  D. Blackwell Comparison of Experiments , 1951 .

[14]  B. Anderson,et al.  Linear Optimal Control , 1971 .

[15]  Howard F. Naish Production Smoothing in the Linear Quadratic Inventory Model , 1994 .

[16]  Michael Woodford,et al.  Optimal Monetary and Fiscal Policy: A Linear-Quadratic Approach , 2003, NBER Macroeconomics Annual.

[17]  Jean Pierre Aubin,et al.  Applied abstract analysis , 1977 .

[18]  Andrew B. Whinston,et al.  A model of decision-making with sequential information-acquisition (part 2) , 1987, Decis. Support Syst..

[19]  S. Liberty,et al.  Linear Systems , 2010, Scientific Parallel Computing.

[20]  J. Bather,et al.  Multi‐Armed Bandit Allocation Indices , 1990 .

[21]  J. Minx,et al.  Climate Change 2014 : Synthesis Report , 2014 .

[22]  William E Rees,et al.  Are We Consuming Too Much—for What? , 2007, Conservation biology : the journal of the Society for Conservation Biology.

[23]  George E. Monahan,et al.  Optimal Stopping in a Partially Observable Markov Process with Costly Information , 1980, Oper. Res..

[24]  Vijay S. Mookerjee,et al.  Purchasing demand information in a stochastic-demand inventory system , 1997 .

[25]  Tamer Basar,et al.  Simultaneous design of measurement and control strategies for stochastic systems with feedback , 1989, Autom..

[26]  A. Wald Foundations of a General Theory of Sequential Decision Functions , 1947 .

[27]  H. Simon,et al.  Dynamic Programming Under Uncertainty with a Quadratic Criterion Function , 1956 .

[28]  C. Striebel Sufficient statistics in the optimum control of stochastic systems , 1965 .

[29]  Manfred SchÄl,et al.  Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal , 1975 .

[30]  Thomas A. Weber,et al.  Fair welfare maximization , 2004 .

[31]  S. L. Shmakov A UNIVERSAL METHOD OF SOLVING QUARTIC EQUATIONS , 2011 .

[32]  Drew Fudenberg,et al.  Dynamic Models of Oligopoly , 1986 .

[33]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[34]  Kaijie Zhu,et al.  Endogenous information acquisition in supply chain management , 2010, Eur. J. Oper. Res..

[35]  Vikram Krishnamurthy,et al.  Algorithms for optimal scheduling and management of hidden Markov model sensors , 2002, IEEE Trans. Signal Process..

[36]  Snorre Lindset,et al.  Optimal information acquisition for a linear quadratic control problem , 2009, Eur. J. Oper. Res..

[37]  Alberto Bemporad,et al.  Robust model predictive control: A survey , 1998, Robustness in Identification and Control.

[38]  Rolf van Dawen,et al.  Negative Dynamic Programming , 1984 .

[39]  A D Tsodikov,et al.  On the optimal policies of cancer screening. , 1991, Mathematical biosciences.

[40]  H. Theil A Note on Certainty Equivalence in Dynamic Planning , 1957 .

[41]  K. Arrow,et al.  Optimal Inventory Policy. , 1951 .

[42]  Robert M. Townsend,et al.  Forecasting the Forecasts of Others , 1983, Journal of Political Economy.

[43]  Peter Lund,et al.  Energy and climate change , 2018 .

[44]  Lones Smith,et al.  The Optimal Level of Experimentation , 2000 .

[45]  Thomas Alois Weber Constrained predictive control for corporate policy , 1997 .

[46]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[47]  W. Fleming Book Review: Discrete-time Markov control processes: Basic optimality criteria , 1997 .

[48]  G. Gallego Scheduling the production of several items with random demands in a single facility , 1990 .

[49]  William S. Lovejoy Technical Note - On the Convexity of Policy Regions in Partially Observed Systems , 1987, Oper. Res..

[50]  R. Friedmann The Reliability of Policy Recommendations and Forecasts from Linear Econometric Models , 1981 .

[51]  K. McCardle Information Acquisition and the Adoption of New Technology , 1985 .

[52]  J. Janssen,et al.  Deterministic and Stochastic Optimal Control , 2013 .

[53]  Alessandro Bonatti,et al.  Menu Pricing and Learning , 2010 .

[54]  Andrew B. Whinston,et al.  A model of decision-making with sequential information-acquisition (Part 1) , 1986, Decis. Support Syst..

[55]  Tamer Basar,et al.  Optimum design of measurement channels and control policies for linear-quadratic stochastic systems , 1994 .

[56]  W. Wonham On the Separation Theorem of Stochastic Control , 1968 .

[57]  Paul R. Milgrom,et al.  Communication and Inventory as Substitutes in Organizing Production , 1988 .

[58]  Michael Athans The Discrete Time Linear-Quadratic-Gaussian Stochastic Control Problem , 1972 .

[59]  C. Sims Implications of rational inattention , 2003 .

[60]  C. C. Holt,et al.  Planning Production, Inventories, and Work Force. , 1962 .

[61]  Thomas A. Weber,et al.  Carbon Markets and Technological Innovation , 2010 .

[62]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[63]  Richard M. Murray,et al.  On a stochastic sensor selection algorithm with applications in sensor scheduling and sensor coverage , 2006, Autom..

[64]  M. Degroot Uncertainty, Information, and Sequential Experiments , 1962 .

[65]  S. Auray,et al.  Dynamic regulation of quality , 2011 .

[66]  Thomas A. Weber,et al.  Population-level intervention and information collection in dynamic healthcare policy , 2017, Health Care Management Science.

[67]  K. West A Variance Bounds Test of the Linear Quadratic Inventory Model , 1985, Journal of Political Economy.

[68]  S Kafandaris,et al.  The Economic Value of Information , 2001, J. Oper. Res. Soc..

[69]  M. Dahleh Laboratory for Information and Decision Systems , 2005 .

[70]  S. Lauritzen Time Series Analysis in 1880: A Discussion of Contributions Made by T.N. Thiele , 1981 .

[71]  L. Meier,et al.  Combined optimum control and estimation theory , 1966 .

[72]  Scott D. Peckham,et al.  Fire as the dominant driver of central Canadian boreal forest carbon balance , 2007, Nature.

[73]  M. Athans On the Determination of Optimal Costly Measurement Strategies for Linear Stochastic Systems , 1972 .