Lifelong Learning for Complementary Generation Control of Interconnected Power Grids With High-Penetration Renewables and EVs

This paper proposes a lifelong learning (LL) based complementary generation control (CGC) of interconnected power grids with high-penetration renewable energy sources and electric vehicles (EVs). The wind farms (WFs), photovoltaic stations (PVs), and EVs are aggregated as a wide-area virtual power plant (WVPP) for automatic generation control (AGC), which can significantly accelerate the system response and reduce the regulation costs of balancing unexpected power mismatches between generation side and demand side. Under such framework, CGC is decomposed into a multi-layer generation command dispatch to rapidly compute an optimal solution. LL is first employed for the primary layer CGC between conventional power plants and a WVPP. Then, the secondary layer is implemented according to the ascending order of regulation costs of all reserve sources. Finally, the tertiary layer is accomplished by a coordinated control in each WFs or PVs, whereas an online optimization of EVs is adopted by considering the charging demands. The imitation learning is introduced to improve the learning efficiency of agents with transfer learning, thus an online optimization of CGC can be satisfied. Case studies are carried out to evaluate the performance of LL for multi-layer CGC of AGC on a practical Hainan power grid of southern China.

[1]  Yu Xichang,et al.  Practical implementation of the SCADA+AGC/ED system of the hunan power pool in the central China power network , 1994 .

[2]  Qinghua Wu,et al.  Nonlinear maximum power point tracking control and modal analysis of DFIG based wind turbine , 2016 .

[3]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[4]  Birgitte Bak-Jensen,et al.  Integration of Vehicle-to-Grid in the Western Danish Power System , 2011, IEEE Transactions on Sustainable Energy.

[5]  A. Yokoyama,et al.  System frequency control by Heat Pump Water Heaters (HPWHs) on customer side based on statistical HPWH model in power system with a large penetration of renewable energy sources , 2010, 2010 International Conference on Power System Technology.

[6]  M. Sami Fadali,et al.  Smart Scheduling and Cost-Benefit Analysis of Grid-Enabled Electric Vehicles for Wind Power Integration , 2014, IEEE Transactions on Smart Grid.

[7]  Tao Yu,et al.  Consensus Transfer ${Q}$ -Learning for Decentralized Generation Command Dispatch Based on Virtual Generation Tribe , 2018, IEEE Transactions on Smart Grid.

[8]  Tomonobu Senjyu,et al.  Fuzzy Control of Distributed PV Inverters/Energy Storage Systems/Electric Vehicles for Frequency Regulation in a Large Power System , 2013, IEEE Transactions on Smart Grid.

[9]  Zhe Chen,et al.  Optimal Operation of Plug-In Electric Vehicles in Power Systems With High Wind Power Penetrations , 2013, IEEE Transactions on Sustainable Energy.

[10]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[11]  Sukumar Mishra,et al.  Exploring frequency control capability of a PV system in a hybrid PV-rotating machine-without storage system , 2014 .

[12]  Xuesong Wang,et al.  Multi-source transfer ELM-based Q learning , 2014, Neurocomputing.

[13]  David C. Yu,et al.  An Economic Dispatch Model Incorporating Wind Power , 2008, IEEE Transactions on Energy Conversion.

[14]  Rahmat-Allah Hooshmand,et al.  Emission, reserve and economic load dispatch problem with non-smooth and non-convex cost functions using the hybrid bacterial foraging-Nelder–Mead algorithm , 2012 .

[15]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[16]  Hsan Hadj Abdallah,et al.  Economic Dispatch for Power System included Wind and Solar Thermal energy , 2009 .

[17]  Liu San-yang,et al.  A smoothing trust-region Newton-CG method for minimax problem , 2008 .

[18]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[19]  Scott D. Sudhoff,et al.  Evolutionary Algorithms for Minimax Problems in Robust Design , 2009, IEEE Transactions on Evolutionary Computation.

[20]  A. Mullane,et al.  An Assessment of the Impact of Wind Generation on System Frequency Control , 2010, IEEE Transactions on Power Systems.

[21]  Jan Peters,et al.  Imitation and Reinforcement Learning , 2010, IEEE Robotics & Automation Magazine.

[22]  Tao Yu,et al.  Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step $Q(\lambda)$ Learning , 2011, IEEE Transactions on Power Systems.

[23]  Sukumar Mishra,et al.  Storage Free Smart Energy Management for Frequency Control in a Diesel-PV-Fuel Cell-Based Hybrid AC Microgrid , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[25]  Bo Yang,et al.  Accelerating bio-inspired optimizer with transfer reinforcement learning for reactive power optimization , 2017, Knowl. Based Syst..

[26]  Luis S. Vargas,et al.  Wind power curtailment and energy storage in transmission congestion management considering power plants ramp rates , 2015, 2015 IEEE Power & Energy Society General Meeting.

[27]  Andrew Lewis,et al.  S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization , 2013, Swarm Evol. Comput..

[28]  D J Burke,et al.  Factors Influencing Wind Energy Curtailment , 2011, IEEE Transactions on Sustainable Energy.

[29]  Tao Yu,et al.  Hierarchical correlated Q-learning for multi-layer optimal generation command dispatch , 2016 .

[30]  N. Jaleeli,et al.  NERC's new control performance standards , 1999 .

[31]  C.D. Vournas,et al.  Application of interruptible contracts to increase wind-power penetration in congested areas , 2004, IEEE Transactions on Power Systems.

[32]  Ibraheem,et al.  Recent philosophies of automatic generation control strategies in power systems , 2005, IEEE Transactions on Power Systems.

[33]  Michael N. Vrahatis,et al.  Particle swarm optimization for minimax problems , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[34]  Q. Henry Wu,et al.  Group Search Optimizer: An Optimization Algorithm Inspired by Animal Searching Behavior , 2009, IEEE Transactions on Evolutionary Computation.

[35]  Le-Ren Chang-Chien,et al.  Modeling of Wind Farm Participation in AGC , 2014, IEEE Transactions on Power Systems.

[36]  L. H. Fink,et al.  Understanding automatic generation control , 1992 .

[37]  Zechun Hu,et al.  Decentralized Vehicle-to-Grid Control for Primary Frequency Regulation Considering Charging Demands , 2013, IEEE Transactions on Power Systems.

[38]  Andreas Sumper,et al.  Participation of wind power plants in system frequency control: Review of grid code requirements and control methods , 2014 .

[39]  Yu Xichang,et al.  Practical implementation of the SCADA+AGC/ED system of the hunan power pool in the central China power network , 1994, IEEE Power Engineering Review.

[40]  Kay Chen Tan,et al.  A New Differential Evolution Algorithm for Minimax Optimization in Robust Design , 2018, IEEE Transactions on Cybernetics.

[41]  Reinaldo A. C. Bianchi,et al.  Transferring knowledge as heuristics in reinforcement learning: A case-based approach , 2015, Artif. Intell..

[42]  Zhao Xu,et al.  Demand as Frequency Controlled Reserve , 2011, IEEE Transactions on Power Systems.

[43]  Taisuke Masuta,et al.  Supplementary Load Frequency Control by Use of a Number of Both Electric Vehicles and Heat Pump Water Heaters , 2012, IEEE Transactions on Smart Grid.

[44]  Yong Zhang,et al.  Uniform Design: Theory and Application , 2000, Technometrics.

[45]  Ning Lu,et al.  An Evaluation of the HVAC Load Potential for Providing Load Balancing Service , 2012, IEEE Transactions on Smart Grid.

[46]  C. Boutilier,et al.  Accelerating Reinforcement Learning through Implicit Imitation , 2003, J. Artif. Intell. Res..

[47]  Mattia Marinelli,et al.  Provision of secondary frequency control via demand response activation on thermostatically controlled loads: Solutions and experiences from Denmark , 2016 .

[48]  Kenji Iba.  Reactive power optimization by genetic algorithm , 1993 .