A Real-time Demand-side Management System Considering User Behavior Using Deep Q-Learning in Home Area Network

In smart grids, demand-side management (DSM) has become an important topic since it can reduce the total electricity cost by smart control and rescheduling of loads, meanwhile, reduce the peak-to-average ratio (PAR) under real-time pricing policy. On the other hand, with the growth of computation ability in recent years and the huge amount of data collected in home area network (HAN), machine learning skills such as reinforcement learning can be well applied into the DSM problem. However, it is hard to determine an optimal energy management strategy since the uncertainty of user behavior and the electricity consumption. In the proposed work, a real-time multi-agent deep reinforcement learning based approach has been proposed to solve the DSM problem in HAN and additionally considers the user behavior to avoid disturbing user comfort, meanwhile, adaptively learns the appliance usage preference and updates the system after each day. The simulation results reveal that the proposed DSM system has improved the energy efficiency in a smart home that not only reduces the electricity cost and peak value but also the PAR value.

[1]  Giovanni Pau,et al.  A Novel Energy Management Approach for Smart Homes Using Bluetooth Low Energy , 2015, IEEE Journal on Selected Areas in Communications.

[2]  Thillainathan Logenthiran,et al.  Demand Side Management in Smart Grid Using Heuristic Optimization , 2012, IEEE Transactions on Smart Grid.

[3]  J. Zico Kolter,et al.  REDD : A Public Data Set for Energy Disaggregation Research , 2011 .

[4]  Xiao-Ping Zhang,et al.  Artificial intelligence based smart energy community management: A reinforcement learning approach , 2019, CSEE Journal of Power and Energy Systems.

[5]  Nadeem Javaid,et al.  A Novel Meta-Heuristic Hybrid Enhanced Differential Harmony Wind Driven (EDHWDO) Optimization Technique for Demand Side Management in Smart Grid , 2018, 2018 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA).

[6]  Pierluigi Siano,et al.  A Review of Architectures and Concepts for Intelligence in Future Electric Energy Systems , 2015, IEEE Transactions on Industrial Electronics.

[7]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[8]  Kelum A. A. Gamage,et al.  Demand side management in smart grid: A review and proposals for future direction , 2014 .

[9]  Geert Deconinck,et al.  Battery Scheduling in a Residential Multi-Carrier Energy System Using Reinforcement Learning , 2018, 2018 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm).

[10]  Jinjun Xiong,et al.  Demand-Side Management of Domestic Electric Water Heaters Using Approximate Dynamic Programming , 2017, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[11]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[12]  Li-Chen Fu,et al.  Demand-side management in residential community realizing sharing economy with bidirectional PEV , 2017, 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[13]  Ssu-li Chang,et al.  The simulation and management of adaptive smart grid oriented power mix based on evolutionary dynamic mechanism under high percentage of renewable energy policy , 2016, 2016 Portland International Conference on Management of Engineering and Technology (PICMET).

[14]  Wanglei,et al.  Intelligent control in smart home based on adaptive neuro fuzzy inference system , 2015, 2015 Chinese Automation Congress (CAC).

[15]  Leïla Merghem,et al.  Demand Side Management in Smart Grids: A Stackelberg Multi Period Multi Provider Game , 2018, 2018 9th IFIP International Conference on New Technologies, Mobility and Security (NTMS).

[16]  George J. Tsekouras,et al.  Optimal Demand-Side Management and Power Generation Scheduling in an All-Electric Ship , 2014, IEEE Transactions on Sustainable Energy.

[17]  Haibo He,et al.  Residential Energy Management with Deep Reinforcement Learning , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[18]  Li-Chen Fu,et al.  Unsupervised context discovery based on hierarchical fusion of heterogeneous features in real smart living environments , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[19]  Li-Chen Fu,et al.  Demand Response in Residential and Commercial Community Considering User Comfort Using Improved Particle Swarm Optimization , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[20]  Nandan Parameswaran,et al.  Flexible power consumption management using Q learning techniques in a smart home , 2013, 2013 IEEE Conference on Clean Energy and Technology (CEAT).

[21]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Multiobjective demand side management solutions for utilities with peak demand deficit , 2014 .

[23]  Eduardo Souto,et al.  User activity recognition for energy saving in smart home environment , 2015, 2015 IEEE Symposium on Computers and Communication (ISCC).

[24]  Bart De Schutter,et al.  Residential Demand Response of Thermostatically Controlled Loads Using Batch Reinforcement Learning , 2017, IEEE Transactions on Smart Grid.