Dynamic programming applications in water resources

The central intention of this survey is to review dynamic programming models for water resource problems and to examine computational techniques which have been used to obtain solutions to these problems. Problem areas surveyed here include aqueduct design, irrigation system control, project development, water quality maintenance, and reservoir operations analysis. Computational considerations impose severe limitation on the scale of dynamic programming problems which can be solved. Inventive numerical techniques for implementing dynamic programming have been applied to water resource problems. Discrete dynamic programming, differential dynamic programming, state incremental dynamic programming, and Howard's policy iteration method are among the techniques reviewed. Attempts have been made to delineate the successful applications, and speculative ideas are offered toward attacking problems which have not been solved satisfactorily.

[1]  John D. C. Little,et al.  The Use of Storage Water in a Hydroelectric System , 1955, Oper. Res..

[2]  David G. Kendall,et al.  Some Problems in the Theory of Dams , 1957 .

[3]  S. Karlin,et al.  Studies in the Mathematical Theory of Inventory and Production, by K.J. Arrow, S. Karlin, H. Scarf with contributions by M.J. Beckmann, J. Gessford, R.F. Muth. Stanford, California, Stanford University Press, 1958, X p.340p., $ 8.75. , 1959, Bulletin de l'Institut de recherches économiques et sociales.

[4]  P. A. P. Moran,et al.  The Theory of Storage , 1961 .

[5]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[6]  N. Buras,et al.  The dynamic programming approach to water‐resources development , 1961 .

[7]  Stuart E. Dreyfus,et al.  Applied Dynamic Programming , 1965 .

[8]  Warren A. Hall,et al.  The optimization of single-purpose reservoir design with the application of dynamic programming to synthetic hydrology samples , 1963 .

[9]  N. Buras,et al.  Conjunctive Operation of Dams and Aquifers , 1963 .

[10]  Onésimo Hernández-Lerma,et al.  Controlled Markov Processes , 1965 .

[11]  H. Halkin.  A Maximum Principle of the Pontryagin Type for Systems Described by Nonlinear Difference Equations , 1966 .

[12]  Walter R. Lynn,et al.  The optimal allocation of stream dissolved oxygen , 1966 .

[13]  R. Bellman.  Dynamic programming. , 1957, Science.

[14]  Warren A. Hall,et al.  Hydropower project output optimization , 1966 .

[15]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[16]  E. H. Lloyd.  Stochastic Reservoir Theory , 1967 .

[17]  G. K. Young.  Finding Reservoir Operating Rules , 1967 .

[18]  C. S. Beightler,et al.  An optimization method for branching multistage water resource systems , 1967 .

[19]  Warren F. Musgrave,et al.  DEVELOPMENT AND ANALYSIS OF INPUT-OUTPUT RELATIONS FOR IRRIGATION WATER , 1967 .

[20]  Z. Schweig,et al.  Optimal Control of Linked Reservoirs , 1968 .

[21]  William S. Butcher,et al.  Optimization of the Operation of a Multiple‐Purpose Reservoir by Dynamic Programming , 1968 .

[22]  William S. Butcher,et al.  Optimal Timing of Irrigation , 1968 .

[23]  J. A. Bather.  A diffusion model for the control of a dam , 1968 .

[24]  C. Revelle,et al.  The Linear Decision Rule in Reservoir Management and Design: 1, Development of the Stochastic Model , 1969 .

[25]  Yacov Y. Haimes,et al.  Dynamic Programing for the Optimal Sequencing of Water Supply Projects , 1969 .

[26]  S. Yakowitz.  Mathematics of adaptive control processes , 1969 .

[27]  Warren A. Hall,et al.  An Alternate Procedure for the Optimization of Operations for Planning with Multiple River, Multiple Purpose Systems , 1969 .

[28]  D. Loucks,et al.  Markov Models for Flow Regulation , 1970 .

[29]  Donald E. Kirk,et al.  Optimal Control Theory , 1970 .

[30]  R. E. Larson,et al.  A dynamic programming successive approximations technique with convergence proofs , 1970 .

[31]  L. Bodin,et al.  Multireservoir Operation Studies , 1970 .

[32]  Thomas L. Morin,et al.  Some Efficient Dynamic Programing Algorithms for the Optimal Sequencing and Scheduling of Water Supply Projects , 1971 .

[33]  William S. Butcher,et al.  STOCHASTIC DYNAMIC PROGRAMMING FOR OPTIMUM RESERVOIR OPERATION1 , 1971 .

[34]  Norman J. Dudley,et al.  Optimal Intraseasonal Irrigation Water Allocation , 1971 .

[35]  Courtney Riordan,et al.  General Multistage Marginal Cost Dynamic Programing Model for the Optimization of a Class of Investment‐Pricing Decisions , 1971 .

[36]  V. T. Chow,et al.  Discrete Differential Dynamic Programing Approach to Water Resources Systems Optimization , 1971 .

[37]  Charles ReVelle,et al.  Operating Rules for Joint Operation of Raw Water Sources , 1971 .

[38]  Nathan Buras.  Scientific allocation of water resources , 1971 .

[39]  E. H. Lloyd,et al.  RESERVOIRS WITH MIXED MARKOVIAN-INDEPENDENT INFLOWS* , 1972 .

[40]  Gregory K. Miller,et al.  Elements of Applied Stochastic Processes , 1972 .

[41]  C. Bradley Russell,et al.  An Optimal Policy for Operating a Multipurpose Reservoir , 1972, Oper. Res..

[42]  L. Eisel.  Chance constrained reservoir model , 1972 .

[43]  David Q. Mayne,et al.  Differential dynamic programming , 1972, The Mathematical Gazette.

[44]  William W.-G. Yeh,et al.  OPTIMAL ALLOCATION OF ARTIFICIAL AERATION ALONG A POLLUTED STREAM USING DYNAMIC PROGRAMMING1 , 1973 .

[45]  Donald Erlenkotter,et al.  Sequencing Expansion Projects , 1970, Oper. Res..

[46]  Norman J. Dudley,et al.  Stochastic reservoir management and system design for irrigation , 1973 .

[47]  Sidney Yakowitz,et al.  A stochastic model for daily river flows in an arid region , 1973 .

[48]  Donald Erlenkotter,et al.  Sequencing of interdependent hydroelectric projects , 1973 .

[49]  Thomas L. Morin.  Pathology of a dynamic programing sequencing algorithm , 1973 .

[50]  W. Trott,et al.  Optimization of Multiple Reservoir System , 1973 .

[51]  Chester C. Kisiel,et al.  Procedures for determining the order of dependence in streamflow records , 1974 .

[52]  Optimal Planning for a Thermal Discharge Treatment System , 1974 .

[53]  Thomas L. Morin.  Optimality of a Heuristic Sequencing Technique , 1974 .

[54]  A. Askew.  Optimum Reservoir Operating Policies and the Imposition of a Reliability Constraint , 1974 .

[55]  Neil S. Grigg,et al.  Minimization of combined sewer overflows by large-scale mathematical programming , 1974, Comput. Oper. Res..

[56]  M. J. Faddy,et al.  Optimal control of finite dams: continuous output procedure , 1974, Advances in Applied Probability.

[57]  S. K. Srinivasan.  Analytic solution of a finite dam governed by a general input , 1974, Journal of Applied Probability.

[58]  Sequential Deterministic Optimization in Reservoir Operation , 1974 .

[59]  W. Yeh,et al.  Optimization of real time operation of a multiple-reservoir system , 1974 .

[60]  Arthur J. Askew.  Chance‐constrained dynamic programing and the optimization of water resource systems , 1974 .

[61]  W. Yeh,et al.  Optimal Timing, Sequencing, and Sizing of Multiple Reservoir Surface Water Supply Facilities , 1974 .

[62]  D. J. White.  Technical Note - Dynamic Programming and Probabilistic Constraints , 1974, Oper. Res..

[63]  P. A. Mawer,et al.  Improved dynamic programing procedures and their practical application to water resource systems , 1974 .

[64]  R. H. Hicks,et al.  A nonlinear programming approach to a very large hydroelectric system optimization , 1974, Math. Program..

[65]  Rolf A. Deininger,et al.  Modeling the regulation of lake superior under uncertainty of future water supplies , 1974 .

[66]  Domhnall T. O'Laoghaire,et al.  Optimal Expansion of a Water Resources System , 1974 .

[67]  David R. Maidment,et al.  Computer time and memory requirements for DP and DDDP in water resource systems analysis , 1975 .

[68]  Arthur J. Askew.  Use of risk premiums in chance-constrained dynamic programing , 1975 .

[69]  S. Arunkumar,et al.  Characterization of optimal operating policies for finite dams , 1975 .

[70]  D. G. Doran.  An efficient transition definition for discrete state reservoir analysis: The divided interval technique , 1975 .

[71]  Neil S. Grigg,et al.  Automatic Control of Large-Scale Combined Sewer Systems , 1975 .

[72]  M. J. Sobel.  Reservoir management models , 1975 .

[73]  Yacov Y. Haimes,et al.  Dynamic Programming for Optimal Capacity Expansion of Wastewater Treatment Plants , 1975 .

[74]  Donald R. Davis,et al.  Comment on ‘Chance‐constrained dynamic programing and optimization of water resource systems’ by Arthur J. Askew , 1975 .

[75]  Warren A. Hall,et al.  OPTIMIZATION OF DESIGN CAPACITY OF AN AQUEDUCT , 1976 .

[76]  L. Mays,et al.  Optimal design of multilevel branching sewer systems , 1976 .

[77]  Optimum control of irrigation water application , 1976 .

[78]  Warren A. Hall,et al.  Aqueduct Optimization with Intermediate Storage , 1976 .

[79]  Sidney Yakowitz,et al.  Small-Sample Hypothesis Tests of Markov Order, with Application to Simulated and Hydrologic Chains , 1976 .

[80]  Arthur J. Askew,et al.  Multilevel incremental dynamic programing , 1976 .

[81]  William W.-G. Yeh,et al.  Operations Models for Central Valley Project , 1976 .

[82]  Norman J. Dudley,et al.  Reliability, trade‐offs, and water resources development modeling with multiple crops , 1976 .

[83]  L. Rossman.  Reliability‐constrained dynamic programing and randomized release rules in reservoir management , 1977 .

[84]  J. W. Cohen,et al.  On Level Crossings and Cycles in Dam Processes , 1977, Math. Oper. Res..

[85]  V. Klemeš.  Discrete representation of storage for stochastic reservoir optimization , 1977 .

[86]  S. Arunkumar,et al.  Optimal regulation policies for a multipurpose reservoir with seasonal input and return function , 1977 .

[87]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Stochastic Control , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[88]  Ronald A. Howard,et al.  Comments on the Origin and Application of Markov Decision Processes , 2002, Oper. Res..

[89]  Some aspects of stochastic reservoir theory — A further comment , 1978 .

[90]  R. Rockafellar,et al.  The Optimal Recourse Problem in Discrete Time: $L^1 $-Multipliers for Inequality Constraints , 1978 .

[91]  A. Askew.  Comments on ‘Reliability‐constrained dynamic programing and randomized release rules in reservoir management’ by Lewis A. Rossman , 1978 .

[92]  J. L. Denny,et al.  Admissible Run-Contingency Type Tests for Independence and Markov Dependence , 1978 .

[93]  S. Arunkumar,et al.  On Optimal Regulation Policies for Certain Multi-Reservoir Systems , 1978, Oper. Res..

[94]  Herman Chernoff,et al.  Optimal Control of a Brownian Motion. , 1978 .

[95]  Martin L. Puterman,et al.  Dynamic Programming and Its Application , 1979 .

[96]  William W.-G. Yeh,et al.  Real-Time Hourly Reservoir Operation , 1979 .

[97]  Sidney Yakowitz,et al.  A nonparametric Markov Model for daily river flow , 1979 .

[98]  S. Yakowitz,et al.  Constrained differential dynamic programming and its application to multireservoir control , 1979 .

[99]  M. Bogle,et al.  Stochastic optimization of a water supply system , 1979 .

[100]  Michael O'Sullivan,et al.  Stochastic optimization of water supply expansion , 1979 .

[101]  Miguel A. Mariño,et al.  Irrigation planning: 2. Water allocation for leaching and irrigation purposes , 1979 .

[102]  M. Sniedovich.  Reliability‐constrained reservoir control problems: 1. Methodological issues , 1979 .

[103]  Robert R. Inman,et al.  Multiobjective dynamic programing with application to a reservoir , 1979 .

[104]  Robert R. Inman,et al.  Multiobjective dynamic programing: A classic problem redressed , 1979 .

[105]  S. Gal.  Optimal management of a multireservoir water supply system , 1979 .

[106]  Leon S. Lasdon,et al.  Feature Article - Survey of Nonlinear Programming Applications , 1980, Oper. Res..

[107]  M. Sniedovich.  A variance‐constrained reservoir control problem , 1980 .

[108]  Economic Model for Reservoir Planning , 1980 .

[109]  U. G. Haussmann,et al.  Some Examples of Optimal Stochastic Controls OR: The Stochastic Maximum Principle at Work , 1981 .

[110]  Sidney J. Yakowitz,et al.  The application of optimal control methodology to nonlinear programming problems , 1981, Math. Program..

[111]  M. Sniedovich.  Some comments on preference order dynamic programming models , 1981 .

[112]  Rafael L. Bras,et al.  The irrigation scheduling problem and evapotranspiration uncertainty , 1981 .