Neural Approximation for the Optimal Control of a Hydroplant with Random Inflows and Concave Revenues

We present a method for computing an approximately optimal policy for the control of a hydroelectric reservoir with random inflows and concave, piecewise linear revenues from electricity sales. Our approach uses neurodynamic programming to approximate the future value function by a neural network. Our approximation architecture, based on the feedforward network, gives very smooth approximate functions, allowing the use of a coarse discretization of the state and inflow variables in the training step of the neural functions. Our model takes into account the head variations on the turbine efficiency and assumes the water flows at a steady rate during each period of the planning horizon, while related models in the literature have made less realistic assumptions of constant head or that the natural inflows were unusable until the next period. Moreover, we extend previous results on the concavity of the expected future rewards as a function of the potential energy in the reservoir and on the structure of optimal decision rules.

[1]  Bernard F. Lamond Stochastic Optimization Of A Hydroelectric Reservoir Using Piecewise Polynomial Approximations , 2003 .

[2]  Christine A. Shoemaker,et al.  Applying Experimental Design and Regression Splines to High-Dimensional Continuous-State Stochastic Dynamic Programming , 1999, Oper. Res..

[3]  Abdeslem Boukhtouta,et al.  Water Reservoir Applications of Markov Decision Processes , 2002 .

[4]  P. Kitanidis,et al.  Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems , 1988 .

[5]  Abdeslem Boukhtouta,et al.  DE TRAVAIL 2002-013 A NEURO-DYNAMIC PROGRAMMING APPROACH FOR STOCHASTIC RESERVOIR MANAGEMENT , 2002 .

[6]  C. S. Buchanan,et al.  Nested Benders decomposition and dynamic programming for reservoir optimisation , 1999, J. Oper. Res. Soc..

[7]  A. Turgeon Optimal operation of multireservoir power systems with stochastic inflows , 1980 .

[8]  André Turgeon A decomposition method for the long-term scheduling of reservoirs in series , 1981 .

[9]  M. J. Sobel,et al.  A reservoir hydroelectric system: Exactly and approximately optimal policies , 1995 .

[10]  B. Lamond,et al.  Optimizing long-term hydro-power production using Markov decision processes , 1996 .

[11]  Ying Li,et al.  Numerical Solution of Continuous-State Dynamic Programs Using Linear and Spline Interpolation , 1993, Oper. Res..

[12]  Lyn C. Thomas,et al.  An aggregate stochastic dynamic programming model of multireservoir systems , 1997 .

[13]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[14]  David J. C. MacKay,et al.  Comparison of Approximate Methods for Handling Hyperparameters , 1999, Neural Computation.

[15]  Daniel P. Heyman,et al.  Stochastic models in operations research , 1982 .

[16]  B. Lamond,et al.  Piecewise affine approximations for the control of a one-reservoir hydroelectric system , 1996 .

[17]  M. V. F. Pereira,et al.  Multi-stage stochastic optimization applied to energy planning , 1991, Math. Program..

[18]  Peter K. Kitanidis,et al.  Improved Dynamic Programming Methods for Optimal Control of Lumped-Parameter Stochastic Systems , 2001, Oper. Res..

[19]  Benjamin Van Roy Neuro-Dynamic Programming: Overview and Recent Trends , 2002 .

[20]  S. Vaida,et al.  Studies in the Mathematical Theory of Inventory and Production , 1958 .

[21]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[22]  A. Turgeon,et al.  An aggregation‐disaggregation approach to long‐term reservoir management , 1998 .

[23]  Bernard F. Lamond,et al.  Lower bounding aggregation and direct computation for an infinite horizon one-reservoir model , 1996 .