Stochastic dynamic programming for reservoir optimal control: Dense discretization and inflow correlation assumption made possible by parallel computing

The solution via dynamic programming (DP) of a reservoir optimal control problem is often computationally prohibitive when the proper description of the inflow process leads to a system model having several state variables and/or when a sufficiently dense state discretization is required to achieve numerical accuracy. Thus, to simplify, the inflow correlation is usually neglected and/or a coarse state discretization is adopted. However, these simplifications may significantly affect the reliability of the solution of the optimization problem. Nowadays, the availability of very powerful computers based on innovative architectures (vector and parallel machines), even in the domain of personal computers (transputer architectures), stimulates the reformulation of the standard dynamic programming algorithm in a form able to exploit these new machine architectures. The reformulated DP algorithm and new machines enable faster and less costly solution of optimization problems involving a system model having two state variables (storage and previous period inflow, then taking into account the inflow correlation) and a number of states (of the order of 104) such as to guarantee a high numerical accuracy.

[1]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[2]  D. White,et al.  Dynamic programming, Markov chains, and the method of successive approximations , 1963 .

[3]  Lotfi A. Zadeh,et al.  Optimality and non-scalar-valued performance criteria , 1963 .

[4]  R. Bellman.  Dynamic programming. , 1957, Science.

[5]  and Charles K. Taft Reswick,et al.  Introduction to Dynamic Systems , 1967 .

[6]  Amedeo R. Odoni,et al.  On Finding the Maximal Gain for Markov Decision Processes , 1969, Oper. Res..

[7]  Rolf A. Deininger,et al.  Generalization of White's Method of Successive Approximations to Periodic Markovian Decision Processes , 1972, Oper. Res..

[8]  Rolf A. Deininger,et al.  Modeling the regulation of lake superior under uncertainty of future water supplies , 1974 .

[9]  D. H. Marks,et al.  A review and evaluation of multiobjective programing techniques , 1975 .

[10]  D. Bertsekas.  Convergence of discretization procedures in dynamic programming , 1975 .

[11]  V. Klemeš.  Discrete representation of storage for stochastic reservoir optimization , 1977 .

[12]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Stochastic Control , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  M. Bogle,et al.  Stochastic optimization of a water supply system , 1979 .

[14]  S. Yakowitz.  Dynamic programming applications in water resources , 1982 .

[15]  R. Bras,et al.  Real time adaptive closed loop control of reservoirs with the High Aswan Dam as a case study , 1983 .

[16]  Udo Schendel,et al.  Introduction to Numerical Methods for Parallel Computers , 1984 .

[17]  B. F. Sule,et al.  Stochastic dynamic programming models for reservoir operation optimization , 1984 .

[18]  William W.-G. Yeh,et al.  Reservoir Management and Operations Models: A State‐of‐the‐Art Review , 1985 .

[19]  H. Loáiciga.  On the use of change constraints in reservoir design and operation modeling , 1988 .

[20]  Rodolfo Soncini-Sessa,et al.  Effects of Reclamation Plans on Water Shortages and Flood Protection , 1988 .

[21]  P. Kitanidis,et al.  Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems , 1988 .

[22]  J. Stedinger,et al.  Sampling stochastic dynamic programming applied to reservoir operation , 1990 .