Deep Reinforcement Learning for Optimal Hydropower Reservoir Operation

AbstractOptimal operation of hydropower reservoir systems is a classical optimization problem of high dimensionality and stochastic nature. A key challenge lies in improving the interpretability of...

[1]  Patrick Willems,et al.  Combining Model Predictive Control with a Reduced Genetic Algorithm for Real-Time Flood Control , 2018 .

[2]  Marcello Restelli,et al.  Tree‐based reinforcement learning for optimal water reservoir operation , 2010 .

[3]  Guangtao Fu,et al.  Cost-Effective River Water Quality Management using Integrated Real-Time Control Technology. , 2017, Environmental science & technology.

[4]  W. Xu,et al.  An Optimal Operation Model for Hydropower Stations Considering Inflow Forecasts with Different Lead-Times , 2018, Water Resources Management.

[5]  Peter Henderson,et al.  An Introduction to Deep Reinforcement Learning , 2018, Found. Trends Mach. Learn..

[6]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[7]  J. D. Quinn,et al.  What Is Controlling Our Control Rules? Opening the Black Box of Multireservoir Operating Policies Using Time‐Varying Sensitivity Analysis , 2019, Water Resources Research.

[8]  Daniel P. Loucks,et al.  Reliability, resiliency, and vulnerability criteria for water resource system performance evaluation , 1982 .

[9]  Yong Peng,et al.  Evaluation of optimization operation models for cascaded hydropower reservoirs to utilize medium range forecasting inflow , 2013 .

[10]  Bo Ming,et al.  Deriving Operating Rules of Pumped Water Storage Using Multiobjective Optimization: Case Study of the Han to Wei Interbasin Water Transfer Project, China , 2017 .

[11]  Qiang Huang,et al.  Simulation with RBF Neural Network Model for Reservoir Operation Rules , 2010 .

[12]  A. I. McLeod,et al.  DIAGNOSTIC CHECKING ARMA TIME SERIES MODELS USING SQUARED‐RESIDUAL AUTOCORRELATIONS , 1983 .

[13]  Thomas G. Robertazzi,et al.  Reinforcement learning based schemes to manage client activities in large distributed control systems , 2019, Physical Review Accelerators and Beams.

[14]  Pedro Ferreira,et al.  An MDP Model-Based Reinforcement Learning Approach for Production Station Ramp-Up Optimization: Q-Learning Analysis , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Guangtao Fu,et al.  Regulatory Implications of Integrated Real-Time Control Technology under Environmental Uncertainty , 2020, Environmental science & technology.

[16]  V. Wolfs,et al.  Real-Time River Flood Control under Historical and Future Climatic Conditions: Flanders Case Study , 2020 .

[17]  William W.-G. Yeh,et al.  Reservoir Management and Operations Models: A State‐of‐the‐Art Review , 1985 .

[18]  Szu-Yin Lin.  Reinforcement learning-based prediction approach for distributed Dynamic Data-Driven Application Systems , 2015, Inf. Technol. Manag..

[19]  R. Vogel,et al.  Decision Trees for Incorporating Hypothesis Tests of Hydrologic Alteration into Hydropower–Ecosystem Tradeoffs , 2020 .

[20]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.

[21]  Ruiqing Niu,et al.  The assessment of landslide susceptibility mapping using random forest and decision tree methods in the Three Gorges Reservoir area, China , 2017, Environmental Earth Sciences.

[22]  Wang Ben-de,et al.  Evaluation of optimization operation models for cascaded hydropower reservoirs to utilize medium range forecasting inflow , 2013 .

[23]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[24]  Peter-Jules van Overloop,et al.  Optimal Real-Time Operation of Multipurpose Urban Reservoirs: Case Study in Singapore , 2014 .

[25]  Nien-Sheng Hsu,et al.  Derived operating rules for a reservoir operation system: Comparison of decision trees, neural decision trees and fuzzy decision trees , 2008 .

[26]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[27]  J. R. Quinlan,et al.  Data Mining Tools See5 and C5.0 , 2004 .

[28]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[29]  Dragan Savic,et al.  Water Reservoir Control with Data Mining , 2003 .

[30]  Andrea Castelletti,et al.  Scalable multi-objective control for large scale water resources systems under uncertainty , 2016 .

[31]  Francesca Pianosi,et al.  Optimal Operation of the Multireservoir System in the Seine River Basin Using Deterministic and Ensemble Forecasts , 2016 .

[32]  P. P. Mujumdar,et al.  A Bayesian Stochastic Optimization Model for a Multi-Reservoir Hydropower System , 2007 .

[33]  Jin-Hee Lee,et al.  Stochastic optimization of multireservoir systems via reinforcement learning , 2007 .

[34]  Andrea Castelletti,et al.  Scalable Multiobjective Control for Large-Scale Water Resources Systems Under Uncertainty , 2018, IEEE Transactions on Control Systems Technology.

[35]  Marcello Restelli,et al.  A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run , 2013 .

[36]  Jay R. Lund,et al.  Optimal Hedging Rule for Reservoir Refill , 2016 .

[37]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[38]  Xiaomang Liu,et al.  Simulating Hydropower Discharge using Multiple Decision Tree Methods and a Dynamical Model Merging Technique , 2020 .

[39]  BenDe Wang,et al.  Inter-basin water transfer-supply model and risk analysis with consideration of rainfall forecast information , 2010 .

[40]  Wei Xu,et al.  A two stage Bayesian stochastic optimization model for cascaded hydropower systems considering varying uncertainty of flow forecasts , 2014 .

[41]  Huicheng Zhou,et al.  Optimization Operation Model Coupled with Improving Water-Transfer Rules and Hedging Rules for Inter-Basin Water Transfer-Supply Systems , 2015, Water Resources Management.

[42]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.