Short term management of hydro-power system using reinforcement learning

The fundamental objective in operation of reservoir complex is to specify an optimal decision policy so that it can maximize the expected value of reward function over the planning horizon. This control problem becomes more challenging as a result of existing different sources of uncertainties that reservoir planner needs to deal with. Usually, a trade-off exists between a value of water in storage and the electricity production. The function on the side of the value of water is uncertain and nonlinear in the reservoir management problem and it heavily depends on storage of reservoir and storage of other reservoirs as well. The challenging task is then how to solve this large-scale multireservoir problem under the presence of several uncertainties. In this thesis, the integration of a novel approach known as Reinforcement Learning (RL) is presented in order to provide an efficient optimization of a large-scale hydroelectric power system. RL is a branch of artificial intelligence method that presents several key benefits in treating problems that are too large to be handled by traditional dynamic programming techniques. In this approach, an agent tries to learn the optimal decision continuously so as to maximize the reward function based on interacting with the environment. This study presents the major concepts and computational aspects of using RL for the short-term planning problem of multireservoir system. The developed reinforcement learning based optimization model was successfully implemented on the Hydro-Quebec multireservoir complex located at the Riviere Romaine, north of the municipality of Havre-Saint-Pierre on the north shore of the St. Lawrence. This model was subsequently used to obtain optimal water release policies for the previously-mentioned reservoir complex. The output of the designed model was compared to the conventional optimization methods known as deterministic dynamic programming. The results show that the RL model is much more efficient and reliable in solving large-scale reservoir operations problems and can give a very good approximate solution to this complex problem.