Multi-agent deep reinforcement learning strategy for distributed energy

Abstract The strong random disturbance issues caused by the large-scale grid connections of distributed energy, such as wind energy, photovoltaic energy storage and electric vehicles, must be resolved. In this paper, we propose a Multi-agent deep reinforcement learning strategy, namely DDQN-CDP, which deeply integrate the improved actor-critic strategy with the neural network. This approach also solves the problem of the lack of continuous action controlling ability of traditional deep reinforcement learning, and obtains an optimal solution by multi-region collaboration. By simulating the modified IEEE standard two-area load frequency control power system model and Hubei power grid model, our results indicate that the proposed strategy can solve the strong random disturbance problem caused by the large-scale grid connections of distributed energy and achieve faster convergence and better control performance than other strategies.

[1]  Lei Xi,et al.  Wolf pack hunting strategy for automatic generation control of an islanding smart distribution network , 2016 .

[2]  Liuqing Yang,et al.  A Virtual Generation Ecosystem Control Strategy for Automatic Generation Control of Interconnected Microgrids , 2020, IEEE Access.

[3]  H. S. Wang,et al.  Cost estimation of plastic injection molding parts through integration of PSO and BP neural network , 2013, Expert Syst. Appl..

[4]  Zhou Bin Reinforcement learning based CPS self-tuning control methodology for interconnected power systems , 2009 .

[5]  Vijay Kumar Singh,et al.  Automatic Generation Control System Using PI and FIS Controller. , 2018, 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT).

[6]  L. H. Fink,et al.  Understanding automatic generation control , 1992 .

[7]  Shouxiang Wang,et al.  A Novel Multi-Agent DDQN-AD Method-Based Distributed Strategy for Automatic Generation Control of Integrated Energy Systems , 2020, IEEE Transactions on Sustainable Energy.

[8]  Liu Xin,et al.  Research on the Automatic Generation Control (AGC) Coordinated Control Modes between Provincial Dispatching and Regional Dispatching in Yunnan Power Grid , 2011 .

[9]  Jie Wu,et al.  Incentive-Driven Deep Reinforcement Learning for Content Caching and D2D Offloading , 2021, IEEE Journal on Selected Areas in Communications.

[10]  Donghai Li,et al.  OPTIMIZATION OF DECENTRALIZED PI/PID CONTROLLERS BASED ON GENETIC ALGORITHM , 2008 .

[11]  Mansour Sheikhan,et al.  RBF neural network based PI pitch controller for a class of 5-MW wind turbines using particle swarm optimization algorithm. , 2012, ISA transactions.

[12]  Tao Yu,et al.  Grouped grey wolf optimizer for maximum power point tracking of doubly-fed induction generator based wind turbine , 2017 .

[13]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[14]  Suraj Pawar,et al.  Distributed deep reinforcement learning for simulation control , 2020, Mach. Learn. Sci. Technol..

[15]  Tao Yu,et al.  Stochastic Optimal CPS Relaxed Control Methodology for Interconnected Power Systems Using Q-Learning Method , 2011 .

[16]  Jie Duan,et al.  Day-Ahead Smart Grid Cooperative Distributed Energy Scheduling With Renewable and Storage Integration , 2016, IEEE Transactions on Sustainable Energy.

[17]  Yimin Zhou,et al.  Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnel , 2017, Energy.

[18]  Hongbin Sun,et al.  Automatic Generation Control Based on Multiple Neural Networks With Actor-Critic Strategy , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Lei Xi,et al.  Multiagent Stochastic Dynamic Game for Smart Generation Control , 2016 .

[20]  Tao Yu,et al.  Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step $Q(\lambda)$ Learning , 2011, IEEE Transactions on Power Systems.

[21]  Shaoyuan Li,et al.  Distributed MPC for Coordinated Energy Efficiency Utilization in Microgrid Systems , 2019, IEEE Transactions on Smart Grid.

[22]  Frank L. Lewis,et al.  Distributed Consensus-Based Economic Dispatch With Transmission Losses , 2014, IEEE Transactions on Power Systems.

[23]  Jing Wang,et al.  PQN and DQN: algorithms for expression microarrays. , 2006, Journal of theoretical biology.

[24]  Goshaidas Ray,et al.  A new approach to the design of robust load-frequency controller for large scale power systems , 1999 .

[25]  Victor M. Zavala,et al.  Large-scale optimal control of interconnected natural gas and electrical transmission systems , 2016 .