The Optimal Regulation of Dams in Continuous Time

The problem of finding optimal release rates for a finite dam is discussed. The dam is fed continuously at a rate, which takes one of a finite number of possible values, transitions between which occur randomly, according to a given transition matrix. The utility associated with any chosen release rate is measured by means of a time-independent concave function. A time-independent policy which determines the release rate, is a vector function of the height of water in the dam, with components corresponding to the possible states of the input variable. Conditions are determined under which such a policy is optimal for the indefinite operation of the dam. That is; at any instant, the expectation of the total utility to be obtained throughout the infinite future, is a maximum. This optimal policy vector satisfies a certain system of nonlinear, first order differential equations. The constant of integration obtained by integrating a particular linear combination of these equations represents the maximum rate of production of utility by the dam. The theory is applied to the case when the utility function is quadratic, and the optimal policy is determined, when there are just two possible input rates. In general explicit solutions are difficult to obtain, but it is shown that any given policy can be transformed into one for which the corresponding rate of production of utility by the dam is increased, and this technique leads to policies which are approximately optimal.