A parallel multi-scenario learning method for near-real-time power dispatch optimization

Abstract Power dispatch problems become more complex when the weight of uncertain renewable resources in the power system gradually increases in recent years. To make use of renewable energy, such as wind energy, more adequately, wisely and intelligently, higher requirements are placed on the level of inter-region power dispatch coordination. In the context, solving the problem of power dispatch on a large scale in near-real-time (5 min in this paper) becomes more important. In this paper, the power dispatch was treated as a sequential decision-making problem and Deep Reinforcement Learning (DRL) with continuous control was introduced to offer a smarter solution. In this way, we designed a novel interactive learning environment based on the economic power dispatch model for the DRL algorithm and we proposed two feasible implementations to handle the different application scenarios. As a result, DRL with a continuous control method has a great performance in our proposed implementations. Moreover, we found that dispatching data richness has a significant influence on the generalization of the learned policy.

[1]  T. P. Imthias Ahamed,et al.  Reinforcement Learning Solution for Unit Commitment Problem through Pursuit Method , 2009 .

[2]  Jasmin E.A.,et al.  A function approximation approach to Reinforcement Learning for solving unit commitment problem with Photo voltaic sources , 2016, 2016 IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES).

[3]  E. A. Jasmin,et al.  Reinforcement Learning approaches to Economic Dispatch problem , 2011 .

[4]  Dipti Srinivasan,et al.  Review of optimization of power dispatch in renewable energy system , 2016, 2016 IEEE Innovative Smart Grid Technologies - Asia (ISGT-Asia).

[5]  D. Draper,et al.  Stochastic Optimization: a Review , 2002 .

[6]  Zheng Li,et al.  A multi-region load dispatch model for the long-term optimum planning of China’s electricity sector , 2017 .

[7]  Gaoxi Xiao,et al.  A Robust Optimization Approach for Energy Generation Scheduling in Microgrids , 2015 .

[8]  Alberto Traverso,et al.  Dynamic economic dispatch using complementary quadratic programming , 2019, Energy.

[9]  Temitope Raphael Ayodele,et al.  Challenges of Grid Integration of Wind Power on Power System Grid Integrity: A Review , 2012 .

[10]  Tao Guo,et al.  An algorithm for combined heat and power economic dispatch , 1996 .

[11]  Damien Ernst,et al.  Reinforcement Learning for Electric Power System Decision and Control: Past Considerations and Perspectives , 2017 .

[12]  Leonardo S. A. Martins,et al.  Advantages of deterministic optimization in long-term hydrothermal scheduling of large-scale power systems , 2013, 2013 IEEE Power & Energy Society General Meeting.

[13]  David C. Yu,et al.  An Economic Dispatch Model Incorporating Wind Power , 2008, IEEE Transactions on Energy Conversion.

[14]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[15]  Bikash Kumar Sahu.  Wind energy developments and policies in China: A short review , 2018 .

[16]  T.M. Athay,et al.  Generation scheduling and control , 1987, Proceedings of the IEEE.

[17]  S. J. Watson,et al.  The optimisation of renewable energy sources in an electrical power system by use of simulation and deterministic planning models , 1996 .

[18]  Alberto Bemporad,et al.  MPC for power systems dispatch based on stochastic optimization , 2014 .

[19]  Jin Lin,et al.  A Versatile Probability Distribution Model for Wind Power Forecast Errors and Its Application in Economic Dispatch , 2013, IEEE Transactions on Power Systems.

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Wei Sun,et al.  A Wasserstein based two-stage distributionally robust optimization model for optimal operation of CCHP micro-grid under uncertainties , 2020 .

[22]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[23]  D.M.V. Kumar.  Intelligent controllers for automatic generation control , 1998, Proceedings of IEEE TENCON '98. IEEE Region 10 International Conference on Global Connectivity in Energy, Computer, Communication and Control (Cat. No.98CH36229).

[24]  Yijin Li,et al.  A learning-based optimization of active power dispatch for a grid-connected microgrid with uncertain multi-type loads , 2017 .

[25]  Takashi Hiyama,et al.  Intelligent Automatic Generation Control , 2011 .

[26]  Hongbo Lian,et al.  Optimal dispatch of virtual power plant using interval and deterministic combined optimization , 2018, International Journal of Electrical Power & Energy Systems.

[27]  S. Najafi,et al.  A New Heuristic Algorithm for Unit Commitment Problem , 2012 .