Wind Farm Power Generation Control Via Double-Network-Based Deep Reinforcement Learning

A model-free deep reinforcement learning (DRL) method is proposed in this article to maximize the total power generation of wind farms through the combination of induction control and yaw control. Specifically, a novel double-network (DN)-based DRL approach is designed to generate control policies for thrust coefficients and yaw angles simultaneously and separately. Two sets of critic-actor networks are constructed to this end. They are linked by a central power-related reward, providing a coordinated control structure while inheriting the critic-actor mechanism's advantages. Compared with conventional DRL methods, the proposed DN-based DRL strategy can adapt to the distinctive and incompatible features of different control inputs, guaranteeing a reliable training process and ensuring superior performance. Also, the prioritized experience replay strategy is utilized to improve the training efficiency of deep neural networks. Simulation tests based on a dynamic wind farm simulator show that the proposed method can significantly increase the power generation for wind farms with different layouts.

[1]  Kincho H. Law,et al.  A data-driven, cooperative wind farm control to maximize the total power production , 2016 .

[2]  Mohd Ashraf Ahmad,et al.  Model-free wind farm control based on random search , 2016, 2016 IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS).

[3]  Nicanor Quijano,et al.  Model-free control for wind farms using a gradient estimation-based algorithm , 2015, 2015 European Control Conference (ECC).

[4]  Wei Hu,et al.  Exploring Deep Reinforcement Learning with Multi Q-Learning , 2016 .

[5]  J. Chow,et al.  Windfarm Power Optimization Using Yaw Angle Control , 2017, IEEE Transactions on Sustainable Energy.

[6]  Jason R. Marden,et al.  A Model-Free Approach to Wind Farm Control Using Game Theoretic Methods , 2013, IEEE Transactions on Control Systems Technology.

[7]  Peng Shi,et al.  Design of Parameter-Self-Tuning Controller Based on Reinforcement Learning for Tracking Noncooperative Targets in Space , 2020, IEEE Transactions on Aerospace and Electronic Systems.

[8]  Jan-Willem van Wingerden,et al.  Adjoint-based model predictive control of wind farms: Beyond the quasi steady-state power maximization , 2017 .

[9]  Shun-ichi Azuma,et al.  A Model-Free Approach for Maximizing Power Production of Wind Farm Using Multi-Resolution Simultaneous Perturbation Stochastic Approximation , 2014 .

[10]  Richard S. Sutton,et al.  Weighted importance sampling for off-policy learning with linear function approximation , 2014, NIPS.

[11]  Aitor Saenz-Aguirre,et al.  Artificial Neural Network Based Reinforcement Learning for Wind Turbine Yaw Control , 2019, Energies.

[12]  Haoran Zhao,et al.  Distributed Model Predictive Control of a Wind Farm for Optimal Active Power ControlPart II: Implementation With Clustering-Based Piece-Wise Affine Wind Turbine Model , 2015, IEEE Transactions on Sustainable Energy.

[13]  Tom Schaul,et al.  Prioritized Experience Replay , 2015, ICLR.

[14]  Jinsong Wu,et al.  Simulation of battery discharge emulator using power electronics device with cascaded P-I control , 2020, 2020 IEEE International Conference on Industrial Technology (ICIT).

[15]  Johan Meyers,et al.  A control-oriented dynamic wind farm model: WFSim , 2017 .

[16]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[17]  Yong Wang,et al.  Differential Evolution With a New Encoding Mechanism for Optimizing Wind Farm Layout , 2018, IEEE Transactions on Industrial Informatics.

[18]  Niko Mittelmeier,et al.  Effects of axial induction control on wind farm energy production - A field test , 2019, Renewable Energy.

[19]  P. Zeng,et al.  Optimization of wind turbine yaw angles in a wind farm using a three-dimensional yawed wake model , 2020 .

[20]  Eunkuk Son,et al.  Blade pitch angle control for aerodynamic performance optimization of a wind farm , 2013 .

[21]  Johan Meyers,et al.  Dynamic Strategies for Yaw and Induction Control of Wind Farms Based on Large-Eddy Simulation and Optimization , 2018 .

[22]  Kincho H. Law,et al.  Cooperative wind turbine control for maximizing wind farm power using sequential convex programming , 2015 .

[23]  Cong Zhang,et al.  A Multi-time Reactive Power Optimization Under Interval Uncertainty of Renewable Power Generation by an Interval Sequential Quadratic Programming Method , 2019, IEEE Transactions on Sustainable Energy.

[24]  Zhao Yang Dong,et al.  Cooperative Wind Farm Control With Deep Reinforcement Learning and Knowledge-Assisted Learning , 2020, IEEE Transactions on Industrial Informatics.

[25]  Jinsong Wu,et al.  A survey of battery energy storage system (BESS), applications and environmental impacts in power systems , 2017, 2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM).

[26]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[27]  Xiong Luo,et al.  Short-Term Wind Speed Forecasting via Stacked Extreme Learning Machine With Generalized Correntropy , 2018, IEEE Transactions on Industrial Informatics.

[28]  J. Meyers,et al.  An optimal control framework for dynamic induction control of wind farms and their interaction with the atmospheric boundary layer , 2017, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[29]  Zhehan Yi,et al.  Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations , 2020, IEEE Transactions on Power Systems.

[30]  David Isele,et al.  Navigating Occluded Intersections with Autonomous Vehicles Using Deep Reinforcement Learning , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Jennifer King,et al.  A Distributed Reinforcement Learning Yaw Control Approach for Wind Farm Energy Capture Maximization* , 2020, 2020 American Control Conference (ACC).

[32]  J. Meyers,et al.  Optimal control of energy extraction in wind-farm boundary layers , 2015, Journal of Fluid Mechanics.

[33]  Pritpal Singh,et al.  Energy Harvesting Technologies: Analysis of their potential for supplying power to sensors in buildings , 2018, 2018 IEEE Third Ecuador Technical Chapters Meeting (ETCM).