Demand Responsive Dynamic Pricing Framework for Prosumer Dominated Microgrids using Multiagent Reinforcement Learning

Demand Response (DR) has a widely recognized potential for improving grid stability and reliability while reducing customers energy bills. However, the conventional DR techniques come with several shortcomings, such as inability to handle operational uncertainties and incurring customer disutility, impeding their wide spread adoption in real-world applications. This paper proposes a new multiagent Reinforcement Learning (RL) based decision-making environment for implementing a Real-Time Pricing (RTP) DR technique in a prosumer dominated microgrid. The proposed technique addresses several shortcomings common to traditional DR methods and provides significant economic benefits to the grid operator and prosumers. To show its better efficacy, the proposed DR method is compared to a baseline traditional operation scenario in a small-scale microgrid system. Finally, investigations on the use of prosumers energy storage capacity in this microgrid highlight the advantages of the proposed method in establishing a balanced market setup.

[1]  Ong Hang See,et al.  A review of residential demand response of smart grid , 2016 .

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Pierluigi Mancarella,et al.  Automated Demand Response From Home Energy Management System Under Dynamic Pricing and Power and Comfort Constraints , 2015, IEEE Transactions on Smart Grid.

[4]  José R. Vázquez-Canteli,et al.  Reinforcement learning for demand response: A review of algorithms and modeling techniques , 2019, Applied Energy.

[5]  Amin Shojaeighadikolaei,et al.  A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids , 2020, 2020 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm).

[6]  Morteza Hashemi,et al.  Energy-Aware Multi-Server Mobile Edge Computing: A Deep Reinforcement Learning Approach , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[7]  Poria Fajri,et al.  A Market Framework for Decentralized Congestion Management in Smart Distribution Grids Considering Collaboration Among Electric Vehicle Aggregators , 2020, IEEE Transactions on Smart Grid.

[8]  Jang-Won Lee,et al.  Residential Demand Response Scheduling With Multiclass Appliances in the Smart Grid , 2016, IEEE Transactions on Smart Grid.

[9]  Pierluigi Siano,et al.  A survey of industrial applications of Demand Response , 2016 .

[10]  Zhao Yang Dong,et al.  Optimal operation of DES/CCHP based regional multi-energy prosumer with demand response , 2016 .

[11]  Henrik Madsen,et al.  Integrating Renewables in Electricity Markets: Operational Problems , 2013 .

[12]  Seung Ho Hong,et al.  A Dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach , 2018, Applied Energy.

[13]  Payman Dehghanian,et al.  Distributed Wind Power Resources for Enhanced Power Grid Resilience , 2019, 2019 North American Power Symposium (NAPS).

[14]  Anamika Dubey,et al.  Aggregation and Bidding of Residential Demand Response into Wholesale Market , 2020, 2020 IEEE Texas Power and Energy Conference (TPEC).

[15]  Amin Kargarian,et al.  Chance-Constrained Microgrid Energy Management with Flexibility Constraints Provided by Battery Storage , 2019, 2019 IEEE Texas Power and Energy Conference (TPEC).

[16]  Chongqing Kang,et al.  Review and prospect of integrated demand response in the multi-energy system , 2017 .

[17]  Amir Abdollahi,et al.  A MILP IGDT-based self-scheduling model for participating in electricity markets , 2016, 2016 24th Iranian Conference on Electrical Engineering (ICEE).

[18]  Seung Ho Hong,et al.  Hour-Ahead Price Based Energy Management Scheme for Industrial Facilities , 2017, IEEE Transactions on Industrial Informatics.

[19]  Mihaela van der Schaar,et al.  Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning , 2016, IEEE Transactions on Smart Grid.

[20]  Vincent W. S. Wong,et al.  Optimal energy consumption scheduling using mechanism design for the future smart grid , 2011, 2011 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[21]  Xinghuo Yu,et al.  Energy-Sharing Provider for PV Prosumer Clusters: A Hybrid Approach Using Stochastic Programming and Stackelberg Game , 2018, IEEE Transactions on Industrial Electronics.

[22]  Shaolei Ren,et al.  Bidirectional Energy Trading and Residential Load Scheduling with Electric Vehicles in the Smart Grid , 2013, IEEE Journal on Selected Areas in Communications.

[23]  Zhihua Qu,et al.  A copula-based Bayesian method for probabilistic solar power forecasting , 2020 .

[24]  Pierluigi Siano,et al.  Demand response and smart grids—A survey , 2014 .

[25]  Christos V. Verikoukis,et al.  A Survey on Demand Response Programs in Smart Grids: Pricing Methods and Optimization Algorithms , 2015, IEEE Communications Surveys & Tutorials.

[26]  Seung Ho Hong,et al.  A real-time decision model for industrial load management in a smart grid , 2016 .