Application of reinforcement learning for energy consumption optimization of district heating system

Heating residential spaces consumed 64 percent of total household energy consumption in Finland. Considering the heat transfer and time delay in the district heating system, the calculation of setpoints of supply temperature requires a comprehensive understanding of the real system, and experienced operators need to manually determine the setpoints. To save energy, a more effective and accurate method is needed for setpoints calculation. In this paper, a reinforcement learning based method is proposed. Through interacting with an Aprosbased simulation model, the agents learn to calculate supply temperature parallelly for lowering energy costs. Simulation results show that the proposed method outperforms the existing method and has the potential to address the problem in real factories. Keywords—district heating, energy consumption optimization, reinforcement learning

[1]  Kuangpu Liu,et al.  A Double-Deck Deep Reinforcement Learning-Based Energy Dispatch Strategy for an Integrated Electricity and District Heating System Embedded with Thermal Inertial and Operational Flexibility , 2022, SSRN Electronic Journal.

[2]  Rishee K. Jain,et al.  Optimizing pipe network design and central plant positioning of district heating and cooling System: A Graph-Based Multi-Objective genetic algorithm approach , 2022, Applied Energy.

[3]  G. Henze,et al.  An optimization framework for the network design of advanced district thermal energy systems , 2022, Energy Conversion and Management.

[4]  Bin Wang,et al.  Exploiting the Flexibility Inside Park-Level Commercial Buildings Considering Heat Transfer Time Delay: A Memory-Augmented Deep Reinforcement Learning Approach , 2022, IEEE Transactions on Sustainable Energy.

[5]  J. Ke,et al.  Energy optimization for regional buildings based on distributed reinforcement learning , 2021, Sustainable Cities and Society.

[6]  Jinkuan Wang,et al.  Multi-agent Deep Reinforcement Learning for Distributed Energy Management and Strategy Optimization of Microgrid Market , 2021 .

[7]  Zhile Yang,et al.  A novel forecasting based scheduling method for household energy management system based on deep reinforcement learning , 2021, Sustainable Cities and Society.

[8]  Elisa Guelpa,et al.  Peak shaving in district heating exploiting reinforcement learning and agent-based modelling , 2021, Eng. Appl. Artif. Intell..

[9]  Marcin Szega,et al.  Short-term scheduling of gas-fired CHP plant with thermal storage using optimization algorithm and forecasting models , 2021 .

[10]  Ahmad Arabkoohsar,et al.  Efficient and cost-effective district heating system with decentralized heat storage units, and triple-pipes , 2019 .

[11]  Ioan Sarbu,et al.  A review of modelling and optimisation techniques for district heating systems , 2019, International Journal of Energy Research.

[12]  Herke van Hoof,et al.  Addressing Function Approximation Error in Actor-Critic Methods , 2018, ICML.

[13]  Scott Bucking,et al.  Distributed evolutionary algorithm for co-optimization of building and district systems for early community energy masterplanning , 2018, Appl. Soft Comput..

[14]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[15]  Sven Werner,et al.  International review of district heating and cooling , 2017 .

[16]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[17]  Satu Paiho,et al.  An analysis of heating energy scenarios of a Finnish case district , 2017 .

[18]  Parham A. Mirzaei,et al.  A Review of District Heating Systems: Modeling and Optimization , 2016, Front. Built Environ..

[19]  J. Schulman,et al.  OpenAI Gym , 2016, ArXiv.

[20]  Tommi A. Karhela,et al.  Using a Digital Twin as the Objective Function for Evolutionary Algorithm Applications in Large Scale Industrial Processes , 2023, IEEE Access.

[21]  S. Werner.  District Heating and Cooling , 2013 .