Multi-Agent Deep Reinforcement Learning for Sectional AGC Dispatch

Aiming at the problem of coordinating system economy, security and control performance in secondary frequency regulation of the power grid, a sectional automatic generation control (AGC) dispatch framework is proposed. The dispatch of AGC is classified as three sections with the sectional dispatch method. Besides, a hierarchical multi-agent deep deterministic policy gradient (HMA-DDPG) algorithm is proposed for the framework in this paper. This algorithm, considering economy and security of the system in AGC dispatch, can ensure the control performance of AGC. Furthermore, through simulation, the control effect of the sectional dispatch method and several AGC dispatch methods on the Guangdong province power grid system and the IEEE 39 bus system is compared. The result shows that the best effect can be achieved with the sectional dispatch method.

[1]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[2]  Tao Yu,et al.  Consensus Transfer ${Q}$ -Learning for Decentralized Generation Command Dispatch Based on Virtual Generation Tribe , 2018, IEEE Transactions on Smart Grid.

[3]  Canbing Li,et al.  Distributed Multi-Energy Operation of Coupled Electricity, Heating, and Natural Gas Networks , 2020, IEEE Transactions on Sustainable Energy.

[4]  L.M.R. Carvalho,et al.  Primal-Dual Interior Point Method Applied to the Short Term Hydroelectric Scheduling Including a Perturbing Parameter , 2009, IEEE Latin America Transactions.

[5]  Tao Yu,et al.  Hierarchical correlated Q-learning for multi-layer optimal generation command dispatch , 2016 .

[6]  Tao Yu,et al.  Culture Evolution Learning for Optimal Carbon-Energy Combined-Flow , 2018, IEEE Access.

[7]  Tao Yu,et al.  Fast Stackelberg equilibrium learning for real-time coordinated energy control of a multi-area integrated energy system , 2019, Applied Thermal Engineering.

[8]  Joshua B. Tenenbaum,et al.  Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation , 2016, NIPS.

[9]  Bo Yang,et al.  Adaptive deep dynamic programming for integrated frequency control of multi-area multi-microgrid systems , 2019, Neurocomputing.

[10]  Canbing Li,et al.  Taxonomy research of artificial intelligence for deterministic solar power forecasting , 2020, Energy Conversion and Management.

[11]  Tao Yu,et al.  Artificial emotional reinforcement learning for automatic generation control of large-scale interconnected power grids , 2017 .

[12]  Tao Yu,et al.  Lifelong Learning for Complementary Generation Control of Interconnected Power Grids With High-Penetration Renewables and EVs , 2018, IEEE Transactions on Power Systems.

[13]  Yitao Liu,et al.  Deep belief network based deterministic and probabilistic wind speed forecasting approach , 2016 .

[14]  Tao Yu,et al.  Optimal Mileage Based AGC Dispatch of a GenCo , 2020, IEEE Transactions on Power Systems.

[15]  Tao Yu,et al.  Stochastic optimal generation command dispatch based on improved hierarchical reinforcement learning approach , 2011 .

[16]  Feng Liu,et al.  Explicit MPC Based on the Galerkin Method for AGC Considering Volatile Generations , 2020, IEEE Transactions on Power Systems.

[17]  Xiaoming Huang,et al.  Adaptive distributed auction-based algorithm for optimal mileage based AGC dispatch with high participation of renewable energy , 2021 .

[18]  Lei Xi,et al.  A wolf pack hunting strategy based virtual tribes control for automatic generation control of smart grid , 2016 .

[19]  Jianchun Peng,et al.  A review of deep learning for renewable energy forecasting , 2019, Energy Conversion and Management.

[20]  W. Yao,et al.  Comprehensive overview of meta-heuristic algorithm applications on PV cell parameter identification , 2020 .