The impact of costly observations and observation delay in stochastic optimal control problems

Practical control problems often present difficulties when observations are needed of the state of a dynamic system, among which are noise, time-lag and the cost involved in obtaining observations. Such problems are treated here through a dynamic programming approach that yields optimal control and optimal observation strategies, the latter of which can be found a priori for ‘linear-quadratic gaussian’ problems. Proper formulation of these problems requires that one make a specific choice for the sequence of events which occurs at each time step, the events being control decision, observation decision and observation reception. This paper examines the range of meaningful event sequences and studies the influence of this factor on the optimal solution. Different event sequences are seen to be natural in different problems. By means of examples, the dependence of the optimal observation strategy on noise and plant characteristics, as well as on observation time-lag, are investigated and certain patterns of ...