Application of two promising Reinforcement Learning algorithms for load shifting in a cooling supply system

Abstract With the increasing use of volatile renewable energies, the requirements for building automation and control systems (BACS) are increasing. Load shifting within local energy systems stabilizes fluctuations in the grid and can be triggered by price signals. The energy purchase can thus be considered and solved as an optimal control problem. Classical approaches, often based on the optimization of mathematical models, are uneconomical in many cases, due to the high effort involved in the model creation. Algorithms from the field of Reinforcement Learning (RL), on the other hand, have a high potential for the automation of energy system optimization, due to their model-free and data-driven characteristics. However, there is still a lack of studies that examine algorithms for BACS-related applications in a structured way. Therefore, we present a study, investigating the potential of two different RL algorithms for load shifting in a cooling supply system. We combine the benefits of Modelica, a powerful modeling language, with RL algorithms and demonstrate how generalized relationships and control decisions can be learned. The case study is modeled according to a cooling supply system in Berlin, Germany. The two different algorithms (DQN and DDPG) are used to control the operation parameters of a central compression chiller, with respect to a price signal. While real monitoring data are used as exogenous influences, the thermal dynamics of the cooling network are simulated. With the learned policies, flexibility in the network is used which leads on average to weekly cost savings of 14 %, compared to direct load coverage. Our results suggest that, under certain conditions, RL is a suitable alternative to established methods. However, we also acknowledge that there are still research questions to address before RL can be applied in real BACS.