The use of intelligent aggregator agents for advanced control of demand response

With the integration of distributed renewable energy sources (DRES) and active demand (e.g., units providing demand response, DR) in the distribution grid, the importance of monitoring the network conditions, managing the line congestions and observing the voltage levels is increasing. The distribution system operator (DSO) needs a mechanism, such as the traffic light system, to screen and approve the proposed operation schedules of the flexible active resources in the distribution grid. Their aggregated control will require the aggregators to employ advanced scheduling algorithms. The DR scheduling algorithms can be set to pursue various goals, for example, maximization of profit or cost reduction, grid support, or provision of the ancillary services. In the paper, we present a new DR scheduling approach suitable for the aggregation agent using approximate Q‐learning (AQL) algorithm scheduling. We present the AQL algorithm and the associated assumptions used in simulations on a real‐world low‐voltage (LV) grid model, comparing the AQL approach results to those of the economic scheduling and the energy scheduling approaches. Our assumption was that the AQL approach could outperform the energy or economic approaches as the AQL agent would be able to learn to avoid the scheduling penalties. The results of our research show that the aggregator agent using the economic approach shows the best economic performance, but causes the most schedule violations. The energy scheduling approach improves the network voltage profile but lowers the aggregator's profit. The AQL approach results in the agent's economic performance between the former two approaches with minimal schedule violations, confirming our research hypothesis. This article is categorized under: Concentrating Solar Power > Systems and Infrastructure Energy Systems Economics > Systems and Infrastructure Energy Infrastructure > Systems and Infrastructure Energy Efficiency > Economics and Policy

[1]  A. F. Gubina,et al.  The flexible demand influence on the joint energy and reserve markets , 2012, 2012 IEEE Power and Energy Society General Meeting.

[2]  Michael D. Sohn,et al.  Characterization of demand response in the commercial, industrial, and residential sectors in the United States , 2016 .

[3]  Tomi Medved,et al.  A traffic light system for enhancing the utilization of demand response in LV distribution networks , 2016, 2016 IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe).

[4]  Robert Entriken,et al.  Storage and demand-side options for integrating wind power , 2014 .

[5]  Damian Flynn,et al.  A new method for determining the demand reserve offer function , 2013 .

[6]  Martin Braun,et al.  Is the distribution grid ready to accept large‐scale photovoltaic deployment? State of the art, progress, and future prospects , 2012 .

[7]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[8]  Lei Yang,et al.  Reinforcement learning for optimal control of low exergy buildings , 2015 .

[9]  M. Shafie‐khah,et al.  A bottom-up approach for demand response aggregators’ participation in electricity markets , 2017 .

[10]  I. Kockar,et al.  Agent-based modeling of the demand-side flexibility , 2011, 2011 IEEE Power and Energy Society General Meeting.

[11]  Daniel J. Veit,et al.  A Critical Survey of Agent-Based Wholesale Electricity Market Models , 2008 .

[12]  Chul-Hwan Kim,et al.  A real-time optimal coordination scheme for the voltage regulation of a distribution network including an OLTC, capacitor banks, and multiple distributed energy resources , 2018 .

[13]  Shan Zhou,et al.  Smart‐grid policies: an international review , 2013 .

[14]  Martin Braun,et al.  Is the distribution grid ready to accept large‐scale photovoltaic deployment? State of the art, progress, and future prospects , 2012 .

[15]  Zheng Wen,et al.  Optimal Demand Response Using Device-Based Reinforcement Learning , 2014, IEEE Transactions on Smart Grid.

[16]  Javier Reneses,et al.  Economic benefits of integrating Active Demand in distribution network planning: A Spanish case study , 2016 .

[17]  Wolf Fichtner,et al.  Agent-based modelling and simulation of smart electricity grids and markets – A literature review , 2016 .

[18]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[19]  Emilio Ghiani,et al.  New electricity distribution network planning approaches for integrating renewable , 2013 .

[20]  Alireza Soroudi,et al.  Decision making under uncertainty in energy systems: state of the art , 2013, ArXiv.

[21]  A. F. Gubina,et al.  Demand-side participation in system reserve provision in a stochastic market model with high wind penetration , 2012, 2012 IEEE International Conference on Power System Technology (POWERCON).

[22]  Goran Andersson,et al.  The role of electric vehicles in smart grids , 2013, Advances in Energy Systems.