Peak shaving in district heating exploiting reinforcement learning and agent-based modelling

Abstract District Heating (DH) technology is considered to be a sustainable and quasi-renewable way of producing and distributing hot water along the city to heat buildings. However, the main obstacle to wider adoption of DH technology is represented by the thermal request peak in the morning hours of winter days, especially in Mediterranean countries. In this paper, this peak-shaving problem is tackled by combining three different approaches. A thermodynamic model is used to monitor the buildings’ thermal response to energy profile modifications. An agent-based model is adopted in order to represent the end-users and their adaptability to variations of temperatures in buildings. Finally, a Reinforcement Learning algorithm is used to optimally mediate between two needs: on the one hand, a set of anticipations and delays is applied to the energy profiles in order to reduce the thermal request peak. On the other hand, the algorithm learns by trial and error the individual agents’ sensitivity to thermal comfort, avoiding drastic modifications for the most sensitive users. The experiments carried out in the DH network in Torino (north-west of Italy) demonstrate that the proposed approach, compared with a literature solution chosen as a baseline, allows to achieve better results in terms of overall performances and speed of convergence.