Control variation as a source of uncertainty: Single input case

This paper presents a theoretical framework and the control strategy for single input discrete-time stochastic systems for which the control variations increase state uncertainty (CVIU systems). This type of system model can be useful in many practical situations, such as in monetary policy problems, medicine and biology, and, in general, in problems for which a complete dynamic model is too complex to be feasible. The optimal control strategy for a single-input CVIU system associated with a convex cost functional is devised using dynamic programming and tools from nonsmooth analysis. Furthermore, this strategy points to a region in the state space in which the optimal action is of no variation, as expected from the cautionary nature of controlling underdetermined systems. In addition, a specific result for the case when the cost functional is differentiable is obtained and discussed. These results are illustrated through a numerical example in economics.

[1]  Xiaohua Xia,et al.  When to initiate HIV therapy: a control theoretic approach , 2003, IEEE Transactions on Biomedical Engineering.

[2]  R. Bellman.  Dynamic programming. , 1957, Science.

[3]  F. Clarke.  Generalized gradients and applications , 1975 .

[4]  T. Vallée,et al.  Optimal versus realized policy rules in a regime-switching framework , 2011 .

[5]  Dimitri P. Bertsekas,et al.  Stochastic optimal control : the discrete time case , 2007 .

[6]  Ádám M. Halász,et al.  Stochastic Modeling and Control of Biological Systems: The Lactose Regulation System of Escherichia Coli , 2008, IEEE Transactions on Automatic Control.

[7]  J.A.M. Felippe de Souza,et al.  Optimal control theory applied to the anti-viral treatment of AIDS , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[8]  H. Kitano.  Systems Biology: A Brief Overview , 2002, Science.

[9]  R. P. Marques,et al.  Discrete-Time Markov Jump Linear Systems , 2004, IEEE Transactions on Automatic Control.

[10]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[11]  Y. P. Liu,et al.  Modeling and estimation for patient controlled analgesia of chronic pain , 1995 .

[12]  Noah Williams,et al.  Monetary Policy with Model Uncertainty: Distribution Forecast Targeting , 2005 .

[13]  A. Banerjee.  Convex Analysis and Optimization , 2006 .

[14]  Andrey V. Savkin,et al.  Application of optimal control theory to analysis of cancer chemotherapy regimens , 2002, Syst. Control. Lett..

[15]  Tamer Basar,et al.  Receding horizon control of jump linear systems and a macroeconomic policy problem , 1999 .

[16]  Alexei Onatski,et al.  ROBUST MONETARY POLICY UNDER MODEL UNCERTAINTY IN A SMALL MODEL OF THE U.S. ECONOMY , 1998, Macroeconomic Dynamics.

[17]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[18]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .