Reinforcement-Learning-Based Intelligent Maximum Power Point Tracking Control for Wind Energy Conversion Systems

This paper proposes an intelligent maximum power point tracking (MPPT) algorithm for variable-speed wind energy conversion systems (WECSs) based on the reinforcement learning (RL) method. The model-free Q-learning algorithm is used by the controller of the WECS to learn a map from states to optimal control actions online by updating the action values according to the received rewards. The experienced action values are stored in a Q-table, based on which the maximum power points (MPPs) are obtained after a certain period of online learning. The learned MPPs are then used to generate an optimum speed-power curve for fast MPPT control of the WECS. Since RL enables the WECS to learn by directly interacting with the environment, knowledge of wind turbine parameters or wind speed information is not required. The proposed MPPT control algorithm is validated by simulation studies for a 1.5-MW doubly-fed induction generator-based WECS and experimental results for a 200-W permanent-magnet synchronous generator-based WECS emulator.

[1]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[2]  Liuchen Chang,et al.  An intelligent maximum power extraction algorithm for inverter-based variable speed wind turbine systems , 2004 .

[3]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  M. Sanada,et al.  Sensorless output maximization control for variable-speed wind generation system using IPMSG , 2003, IEEE Transactions on Industry Applications.

[5]  G. Tapia,et al.  Modeling and control of a wind turbine driven doubly fed induction generator , 2003 .

[6]  Hui Li,et al.  Neural-network-based sensorless maximum wind energy capture with compensated power coefficient , 2004, IEEE Transactions on Industry Applications.

[7]  Yongzheng Zhang,et al.  Sensorless Maximum Power Point Tracking of Wind by DFIG Using Rotor Position Phase Lock Loop (PLL) , 2009, IEEE Transactions on Power Electronics.

[8]  R.G. Harley,et al.  Wind Speed Estimation Based Sensorless Output Maximization Control for a Wind Turbine Driving a DFIG , 2008, IEEE Transactions on Power Electronics.

[9]  Chee Wei Tan,et al.  A review of maximum power point tracking algorithms for wind energy systems , 2012 .

[10]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[11]  Barry W. Williams,et al.  Wind Turbine Power Coefficient Analysis of a New Maximum Power Point Tracking Technique , 2013, IEEE Transactions on Industrial Electronics.

[12]  Xavier Roboam,et al.  Architecture Complexity and Energy Efficiency of Small Wind Turbines , 2007, IEEE Transactions on Industrial Electronics.

[13]  L. Buşoniu,et al.  A comprehensive survey of multi-agent reinforcement learning , 2011 .

[14]  Gonzalo Abad,et al.  Experimental evaluation of wind turbines maximum power point tracking controllers , 2006 .

[15]  Rafael Wisniewski,et al.  Estimation of Rotor Effective Wind Speed: A Comparison , 2013, IEEE Transactions on Control Systems Technology.

[16]  Dong-Choon Lee,et al.  MPPT Control of Wind Generation Systems Based on Estimated Wind Speed Using SVR , 2008, IEEE Transactions on Industrial Electronics.

[17]  V. T. Ranganathan,et al.  A Method of Tracking the Peak Power Points for a Variable Speed Wind Energy Conversion System , 2002, IEEE Power Engineering Review.

[18]  Xu Yang,et al.  Wind Speed and Rotor Position Sensorless Control for Direct-Drive PMG Wind Turbines , 2010, IEEE Transactions on Industry Applications.

[19]  R. Cardenas,et al.  Sensorless vector control of induction machines for variable-speed wind energy applications , 2004, IEEE Transactions on Energy Conversion.

[20]  Kostas Kalaitzakis,et al.  Design of a maximum power tracking system for wind-energy-conversion applications , 2006, IEEE Transactions on Industrial Electronics.

[21]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[22]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[23]  Jie Chen,et al.  On Optimizing the Transient Load of Variable-Speed Wind Energy Conversion System During the MPP Tracking Process , 2014, IEEE Transactions on Industrial Electronics.

[24]  Yishay Mansour,et al.  Learning Rates for Q-learning , 2004, J. Mach. Learn. Res..

[25]  Maurizio Cirrincione,et al.  Neural MPPT Control of Wind Generators With Induction Machines Without Speed Sensors , 2011, IEEE Transactions on Industrial Electronics.

[26]  Hai-Jiao Guo,et al.  A Novel Algorithm for Fast and Efficient Speed-Sensorless Maximum Power Point Tracking in Wind Energy Conversion Systems , 2011, IEEE Transactions on Industrial Electronics.

[27]  Roberto Cárdenas,et al.  Overview of control systems for the operation of DFIGs in wind energy applications , 2013, IECON 2013 - 39th Annual Conference of the IEEE Industrial Electronics Society.

[28]  Wei Qiao,et al.  A Review on Position/Speed Sensorless Control for Permanent-Magnet Synchronous Machine-Based Wind Energy Conversion Systems , 2013, IEEE Journal of Emerging and Selected Topics in Power Electronics.

[29]  Yilmaz Sozer,et al.  Stability Analysis of Maximum Power Point Tracking (MPPT) Method in Wind Power Systems , 2013 .