Multi-agent Deep Reinforcement Learning for Zero Energy Communities

A Zero Energy Building (ZEB) has its net energy usage over a period of one year as zero, i.e., its energy use is not larger than its overall renewables generation. A collection of such ZEBs forms a Zero Energy Community (ZEC). This paper addresses the problem of energy sharing in such a community. This is different from previously addressed energy sharing between buildings as our focus is on the improvement of community energy status, while traditionally research focused on reducing losses due to transmission and storage, or achieving economic gains. We model this problem in a multi-agent environment and propose a Deep Reinforcement Learning (DRL) based solution. Results indicate that with time buildings learn to collaborate and learn a policy comparable to the optimal policy, which in turn improves the ZEC's energy status. Buildings with no renewables preferred to request energy from their neighbours rather than from the supply grid.

[1]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[2]  Prashant J. Shenoy,et al.  Sharing renewable energy in smart microgrids , 2013, 2013 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS).

[3]  Damien Ernst,et al.  Deep Reinforcement Learning Solutions for Energy Microgrids Management , 2016 .

[4]  Peter Stone,et al.  Reinforcement Learning for RoboCup Soccer Keepaway , 2005, Adapt. Behav..

[5]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[6]  Gerald Tesauro,et al.  Reinforcement Learning in Autonomic Computing: A Manifesto and Case Studies , 2007, IEEE Internet Computing.

[7]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[8]  Genku Kayo,et al.  Energy sharing and matching in different combinations of buildings, CHP capacities and operation strategy , 2014 .

[9]  Wil L. Kling,et al.  Comfort-constrained demand flexibility management for building aggregations using a decentralized approach , 2015, 2015 International Conference on Smart Cities and Green ICT Systems (SMARTGREENS).

[10]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[11]  Siobhán Clarke,et al.  Residential demand response: Experimental evaluation and comparison of self-organizing techniques , 2017 .

[12]  Zbigniew Nahorski,et al.  Dynamic Pricing and Balancing Mechanism for a Microgrid Electricity Market , 2014, IEEE Conf. on Intelligent Systems.

[13]  D. Barrios-Aranibar,et al.  LEARNING FROM DELAYED REWARDS USING INFLUENCE VALUES APPLIED TO COORDINATION IN MULTI-AGENT SYSTEMS , 2007 .

[14]  P. Sanjeevikumar,et al.  Investigations of Microgrid Stability and Optimum Power Sharing Using Robust Control of Grid Tie PV Inverter , 2018 .

[15]  Martin A. Riedmiller,et al.  Karlsruhe Brainstormers - A Reinforcement Learning Approach to Robotic Soccer , 2000, RoboCup.

[16]  Ting Zhu,et al.  iDES: Incentive-driven distributed energy sharing in sustainable microgrids , 2014, International Green Computing Conference.

[17]  Celson Lima,et al.  A cooperative net zero energy community to improve load matching , 2016 .

[18]  Richard S. Sutton,et al.  A Deeper Look at Experience Replay , 2017, ArXiv.

[19]  Zifa LIU,et al.  Distributed reinforcement learning to coordinate current sharing and voltage restoration for islanded DC microgrid , 2018 .

[20]  Claudio A. Canizares,et al.  A centralized optimal energy management system for microgrids , 2011, 2011 IEEE Power and Energy Society General Meeting.

[21]  Antonio Liotta,et al.  On-Line Building Energy Optimization Using Deep Reinforcement Learning , 2017, IEEE Transactions on Smart Grid.

[22]  Vinny Cahill,et al.  Multi-agent residential demand response based on load forecasting , 2013, 2013 1st IEEE Conference on Technologies for Sustainability (SusTech).

[23]  P. Torcellini,et al.  Criteria for Definition of Net Zero Energy Buildings , 2010 .