A Self-Governed Online Energy Management and Trading for Smart Micro/Nano-Grids

Joint energy consumption and trading management is still a major challenge in smart (micro) grids. The main goal of solving such problems is to flatten the aggregate power consumption-generation curve and increase the local direct power trading among the participants as much as possible. Here, an inclusive formulation for energy management and trading of a micro/nano-grid (M/NG) is proposed in this article. Subsequently, a holistic solution to jointly optimizing the internal energy consumption management and external local energy trading for a smart grid including several M/NGs is provided. As the problem is computationally intractable, the proposed approach involves three hierarchical stages. First, a game-theoretic online stochastic energy management model is provided with a reinforcement learning solution by which the M/NGs can schedule their power consumptions. Second, an effective incentive-compatible double-auction is formulated by which the M/NGs can directly trade with each other. Third, the central controller develops an optimal power allocation program to reduce the power transmission loss and the destructive effects of local energy trading. The simulation results validate the efficiency of the proposed framework.

[1]  Jianwei Huang,et al.  Incentivizing Energy Trading for Interconnected Microgrids , 2016, IEEE Transactions on Smart Grid.

[2]  Cheng Wang,et al.  Energy-Sharing Model With Price-Based Demand Response for Microgrids of Peer-to-Peer Prosumers , 2017, IEEE Transactions on Power Systems.

[3]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[4]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[5]  Oriol Gomis-Bellmunt,et al.  Trends in Microgrid Control , 2014, IEEE Transactions on Smart Grid.

[6]  Xinghuo Yu,et al.  Energy-Sharing Provider for PV Prosumer Clusters: A Hybrid Approach Using Stochastic Programming and Stackelberg Game , 2018, IEEE Transactions on Industrial Electronics.

[7]  Daniel Friedman,et al.  The Double Auction Market : Institutions, Theories, And Evidence , 2018 .

[8]  Cheng Wang,et al.  Energy Sharing Management for Microgrids With PV Prosumers: A Stackelberg Game Approach , 2017, IEEE Transactions on Industrial Informatics.

[9]  Mihaela van der Schaar,et al.  Dynamic Pricing and Energy Consumption Scheduling With Reinforcement Learning , 2016, IEEE Transactions on Smart Grid.

[10]  Alan Scheller-Wolf,et al.  Design of a Multi–Unit Double Auction E–Market , 2002, Comput. Intell..

[11]  Fuhito Kojima,et al.  Double auction with interdependent values: Incentives and efficiency , 2017 .

[12]  Zechun Hu,et al.  Distributed Coordination of EV Charging With Renewable Energy in a Microgrid of Buildings , 2018, IEEE Transactions on Smart Grid.

[13]  Günther Palm,et al.  Value-Difference Based Exploration: Adaptive Control between Epsilon-Greedy and Softmax , 2011, KI.

[14]  M. Hadi Amini,et al.  A Decentralized Trading Algorithm for an Electricity Market with Generation Uncertainty , 2017, ArXiv.

[15]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[16]  Xi Fang,et al.  3. Full Four-channel 6.3-gb/s 60-ghz Cmos Transceiver with Low-power Analog and Digital Baseband Circuitry 7. Smart Grid — the New and Improved Power Grid: a Survey , 2022 .

[17]  P. Klemperer,et al.  Supply Function Equilibria in Oligopoly under Uncertainty , 1989 .

[18]  Xu Andy Sun,et al.  Adaptive Robust Optimization for the Security Constrained Unit Commitment Problem , 2013, IEEE Transactions on Power Systems.

[19]  J. Mohorko,et al.  Single-hop vs. Multi-hop - Energy efficiency analysis in wireless sensor networks , 2010 .

[20]  Rodrigo Palma-Behnke,et al.  Energy management system for a renewable based microgrid with a demand side management mechanism , 2011, 2011 IEEE Symposium on Computational Intelligence Applications In Smart Grid (CIASG).

[21]  Maryam Kamgarpour,et al.  Game Theoretic Analysis of Auction Mechanisms Modeled by Constrained Optimization Problems , 2017, ArXiv.

[22]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[23]  M. Shahidehpour,et al.  Price-based unit commitment: a case of Lagrangian relaxation versus mixed integer programming , 2005, IEEE Transactions on Power Systems.

[24]  Jianwei Huang,et al.  An Online Learning Algorithm for Demand Response in Smart Grid , 2018, IEEE Transactions on Smart Grid.

[25]  Vincent W. S. Wong,et al.  Autonomous Demand-Side Management Based on Game-Theoretic Energy Consumption Scheduling for the Future Smart Grid , 2010, IEEE Transactions on Smart Grid.

[26]  Qiang Fu,et al.  Microgrid Generation Capacity Design With Renewables and Energy Storage Addressing Power Quality and Surety , 2012, IEEE Transactions on Smart Grid.

[27]  Tingwen Huang,et al.  Reinforcement Learning in Energy Trading Game Among Smart Microgrids , 2016, IEEE Transactions on Industrial Electronics.

[28]  Azam Khalili,et al.  Fully Distributed Demand Response Using the Adaptive Diffusion–Stackelberg Algorithm , 2017, IEEE Transactions on Industrial Informatics.

[29]  M. Satterthwaite,et al.  Efficient Mechanisms for Bilateral Trading , 1983 .

[30]  Guohong Wu,et al.  Optimal operation planning method for isolated micro grid considering uncertainties of renewable power generations and load demand , 2012, IEEE PES Innovative Smart Grid Technologies.

[31]  Xiaofeng Liao,et al.  Reinforcement Learning for Constrained Energy Trading Games With Incomplete Information , 2017, IEEE Transactions on Cybernetics.

[32]  Chao Yang,et al.  Auction Mechanisms for Energy Trading in Multi-Energy Systems , 2018, IEEE Transactions on Industrial Informatics.

[33]  P. Pinson,et al.  Exogenous Cost Allocation in Peer-to-Peer Electricity Markets , 2019, IEEE Transactions on Power Systems.

[34]  Hussein T. Mouftah,et al.  A distributed game theoretic approach to energy trading in the smart grid , 2015, 2015 IEEE Electrical Power and Energy Conference (EPEC).

[35]  H. Farhangi,et al.  The path of the smart grid , 2010, IEEE Power and Energy Magazine.

[36]  Walid Saad,et al.  A Game-Theoretic Approach to Energy Trading in the Smart Grid , 2013, IEEE Transactions on Smart Grid.

[37]  Yoav Shoham,et al.  Multiagent Systems - Algorithmic, Game-Theoretic, and Logical Foundations , 2009 .

[38]  Liang Xiao,et al.  Reinforcement Learning-based Energy Trading for Microgrids , 2018, ArXiv.

[39]  Ming Zeng,et al.  An Incentivized Auction-Based Group-Selling Approach for Demand Response Management in V2G Systems , 2015, IEEE Transactions on Industrial Informatics.

[40]  Farrokh Albuyeh,et al.  Grid of the future , 2009, IEEE Power and Energy Magazine.

[41]  Walid Saad,et al.  Managing Price Uncertainty in Prosumer-Centric Energy Trading: A Prospect-Theoretic Stackelberg Game Approach , 2017, IEEE Transactions on Smart Grid.

[42]  Walid Saad,et al.  A noncooperative game for double auction-based energy trading between PHEVs and distribution grids , 2011, 2011 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[43]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[44]  M. Hadi Amini,et al.  A Decentralized Framework for Real-Time Energy Trading in Distribution Networks with Load and Generation Uncertainty , 2017, ArXiv.

[45]  Vincent W. S. Wong,et al.  Load Scheduling and Power Trading in Systems With High Penetration of Renewable Energy Resources , 2016, IEEE Transactions on Smart Grid.

[46]  Yishay Mansour,et al.  Learning Rates for Q-learning , 2004, J. Mach. Learn. Res..

[47]  Rahul Mohanani,et al.  Demand side energy management in hybrid microgrid system using heuristic techniques , 2016, 2016 IEEE International Conference on Industrial Technology (ICIT).

[48]  Steven R. Williams,et al.  The Double Auction Market: Institutions , 1991 .

[49]  Munther A. Dahleh,et al.  Demand Response Using Linear Supply Function Bidding , 2015, IEEE Transactions on Smart Grid.

[50]  Peter Palensky,et al.  Demand Side Management: Demand Response, Intelligent Energy Systems, and Smart Loads , 2011, IEEE Transactions on Industrial Informatics.