A Novel Multi-Agent DDQN-AD Method-Based Distributed Strategy for Automatic Generation Control of Integrated Energy Systems

The widely adoption of distributed renewable energy sources (DREs) effectively reduces carbon emission and beat atmospheric haze in developing countries. However, random disturbance issues emerge in power grids with DREs when applying traditional centralized automatic generation control (AGC) strategies. Therefore, a multi-agent distributed control strategy is proposed for AGC in this article, which is mainly based on the concept of deep reinforcement learning, and developed by the strategy of action discovery. Moreover, area control error and the amount of carbon emission are employed in reward functions to obtain optimal solutions in the implementing process of the proposed strategy. Simulations are provided in the work to show the effectiveness of the strategy, while comparisons are also offered, where the simulating results obtained by two other intelligent AGC algorithms are used as references, according to which, the superiority of the proposed strategy is confirmed.

[1]  Ali Feliachi,et al.  NERC compliant decentralized load frequency control design using model predictive control , 2003, 2003 IEEE Power Engineering Society General Meeting (IEEE Cat. No.03CH37491).

[2]  Bikramjit Banerjee,et al.  Reinforcement Learning with Action Discovery , 2009 .

[3]  Tao Yu,et al.  Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step $Q(\lambda)$ Learning , 2011, IEEE Transactions on Power Systems.

[4]  Tao Yu,et al.  Distributed multi-step Q(λ) learning for Optimal Power Flow of large-scale power grids , 2012 .

[5]  Lei Xi,et al.  A novel multi-agent decentralized win or learn fast policy hill-climbing with eligibility trace algorithm for smart generation control of interconnected complex power grids☆ , 2015 .

[6]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[7]  Yitao Liu,et al.  Deep belief network based deterministic and probabilistic wind speed forecasting approach , 2016 .

[8]  Hassan A. Yousef,et al.  Load frequency control of a multi-area power system with PV penetration: PI and PID approach in presence of time delay , 2016, 2016 51st International Universities Power Engineering Conference (UPEC).

[9]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[10]  Lei Xi,et al.  A wolf pack hunting strategy based virtual tribes control for automatic generation control of smart grid , 2016 .

[11]  Sukumar Mishra,et al.  Storage Free Smart Energy Management for Frequency Control in a Diesel-PV-Fuel Cell-Based Hybrid AC Microgrid , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Victor M. Zavala,et al.  Large-scale optimal control of interconnected natural gas and electrical transmission systems , 2016 .

[13]  Lei Xi,et al.  Wolf pack hunting strategy for automatic generation control of an islanding smart distribution network , 2016 .

[14]  Enrico Anderlini,et al.  Control of a Point Absorber Using Reinforcement Learning , 2016, IEEE Transactions on Sustainable Energy.

[15]  Jaewan Suh,et al.  Flexible Frequency Operation Strategy of Power System With High Renewable Penetration , 2017, IEEE Transactions on Sustainable Energy.

[16]  Jianfeng Chen,et al.  Smart generation control based on multi-agent reinforcement learning with the idea of the time tunnel , 2017 .

[17]  Venkata Dinavahi,et al.  Direct Interval Forecast of Uncertain Wind Power Based on Recurrent Neural Networks , 2018, IEEE Transactions on Sustainable Energy.

[18]  Yitao Liu,et al.  Deep Learning-Based Interval State Estimation of AC Smart Grids Against Sparse Cyber Attacks , 2018, IEEE Transactions on Industrial Informatics.

[19]  Anindya Das,et al.  Robust automatic generation control in two area thermal-hydro-nuclear plant with 2DOFPID controller , 2018, 2018 Technologies for Smart-City Energy Security and Power (ICSESP).

[20]  Lei Xi,et al.  A Novel Automatic Generation Control Method Based on the Ecological Population Cooperative Control for the Islanded Smart Grid , 2018, Complex..

[21]  Andreas Spanias,et al.  Optimizing Kernel Machines Using Deep Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Lei Xi,et al.  Research on Hierarchical and Distributed Control for Smart Generation Based on Virtual Wolf Pack Strategy , 2018, Complex..

[23]  Vijay Kumar Singh,et al.  Automatic Generation Control System Using PI and FIS Controller. , 2018, 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT).

[24]  Peter Andras,et al.  High-Dimensional Function Approximation With Neural Networks for Large Volumes of Data , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Hao Tang,et al.  Simulation Model for the AGC System of Isolated Microgrid Based on Q-learning Method , 2018, 2018 IEEE 7th Data Driven Control and Learning Systems Conference (DDCLS).

[26]  Mohammad Javad Yazdanpanah,et al.  Distributed Optimal Microgrid Energy Management With Considering Stochastic Load , 2019, IEEE Transactions on Sustainable Energy.

[27]  Zhenhua Li,et al.  Inter-harmonic parameters estimation in power grid based on accelerated PSO and T5R11 window , 2019 .

[28]  Tian Tian,et al.  Optimal Management for Grid-Connected Three/Single-Phase Hybrid Multimicrogrids , 2020, IEEE Transactions on Sustainable Energy.

[29]  Ran Chen,et al.  Coordinated Control of Passive Transition from Grid-Connected to Islanded Operation for Three/Single-Phase Hybrid Multimicrogrids Considering Speed and Smoothness , 2020, IEEE Transactions on Industrial Electronics.