An Energy Management System at the Edge based on Reinforcement Learning

In this work, we propose an IoT edge-based energy management system devoted to minimizing the energy cost for the daily-use of in-home appliances. The proposed approach employs a load scheduling based on a load shifting technique, and it is designed to operate in an edge-computing environment naturally. The scheduling considers all together time-variable profiles for energy cost, energy production, and energy consumption for each shiftable appliance. Deadlines for load termination can also be expressed. In order to address these goals, the scheduling problem is formulated as a Markov decision process and then processed through a reinforcement learning technique. The approach is validated by the development of an agent-based real-world test case deployed in an edge context.

[1]  Franco Cicirelli,et al.  Metamodeling of Smart Environments: from design to implementation , 2017, Adv. Eng. Informatics.

[2]  Xiaoqing Han,et al.  Review on the research and practice of deep learning and reinforcement learning in smart grids , 2018, CSEE Journal of Power and Energy Systems.

[3]  Carlo Mastroianni,et al.  A Unified Model for the Optimal Management of Electrical and Thermal Equipment of a Prosumer in a DR Environment , 2019, IEEE Transactions on Smart Grid.

[4]  Yuanzhang Xiao,et al.  Deep Reinforcement Learning Based Residential Demand Side Management With Edge Computing , 2019, 2019 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm).

[5]  N. Amjady,et al.  Energy price forecasting - problems and proposals for such predictions , 2006 .

[6]  Peter Palensky,et al.  Demand Side Management: Demand Response, Intelligent Energy Systems, and Smart Loads , 2011, IEEE Transactions on Industrial Informatics.

[7]  Joern Ploennigs,et al.  Materializing the Promises of Cognitive IoT: How Cognitive Buildings Are Shaping the Way , 2018, IEEE Internet of Things Journal.

[8]  Yu-Chen Hu,et al.  Residential Consumer-Centric Demand-Side Management Based on Energy Disaggregation-Piloting Constrained Swarm Intelligence: Towards Edge Computing , 2018, Sensors.

[9]  Franco Cicirelli,et al.  Thermal comfort management leveraging deep reinforcement learning and human-in-the-loop , 2020, 2020 IEEE International Conference on Human-Machine Systems (ICHMS).

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Franco Cicirelli,et al.  A Cognitive Enabled, Edge-Computing Architecture for Future Generation IoT Environments , 2019, 2019 IEEE 5th World Forum on Internet of Things (WF-IoT).

[12]  Weisong Shi,et al.  Edge Computing: Vision and Challenges , 2016, IEEE Internet of Things Journal.

[13]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[14]  Koshy George,et al.  A comparison of reinforcement learning based approaches to appliance scheduling , 2016, 2016 2nd International Conference on Contemporary Computing and Informatics (IC3I).

[15]  E. A. Jasmin,et al.  Residential Load Scheduling With Renewable Generation in the Smart Grid: A Reinforcement Learning Approach , 2019, IEEE Systems Journal.

[16]  Barbara Messing,et al.  An Introduction to MultiAgent Systems , 2002, Künstliche Intell..

[17]  Amjad Anvari-Moghaddam,et al.  Demand Side Management Using the Internet of Energy Based on Fog and Cloud Computing , 2017, 2017 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[18]  Enrico Pontelli,et al.  A Multiagent System Approach to Scheduling Devices in Smart Homes , 2017, AAMAS.

[19]  Antonio Iera,et al.  Edge Computing and Social Internet of Things for Large-Scale Smart Environments Development , 2018, IEEE Internet of Things Journal.

[20]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[21]  Franco Cicirelli,et al.  ITEMa: A methodological approach for cognitive edge computing IoT ecosystems , 2019, Future Gener. Comput. Syst..

[22]  Diane J. Cook,et al.  Designing and modeling smart environments , 2006, 2006 International Symposium on a World of Wireless, Mobile and Multimedia Networks(WoWMoM'06).

[23]  Ahmed K. Noor,et al.  Potential of Cognitive Computing and Cognitive Systems , 2014 .

[24]  Marco Levorato,et al.  Residential Demand Response Using Reinforcement Learning , 2010, 2010 First IEEE International Conference on Smart Grid Communications.