A Novel Deep Learning Backstepping Controller-Based Digital Twins Technology for Pitch Angle Control of Variable Speed Wind Turbine

This paper proposes a deep deterministic policy gradient (DDPG) based nonlinear integral backstepping (NIB) in combination with model free control (MFC) for pitch angle control of variable speed wind turbine. In particular, the controller has been presented as a digital twin (DT) concept, which is an increasingly growing method in a variety of applications. In DDPG-NIB-MFC, the pitch angle is considered as the control input that depends on the optimal rotor speed, which is usually derived from effective wind speed. The system stability according to the Lyapunov theory can be achieved by the recursive nature of the backstepping theory and the integral action has been used to compensate for the steady-state error. Moreover, due to the nonlinear characteristics of wind turbines, the MFC aims to handle the un-modeled system dynamics and disturbances. The DDPG algorithm with actor-critic structure has been added in the proposed control structure to efficiently and adaptively tune the controller parameters embedded in the NIB controller. Under this effort, a digital twin of a presented controller is defined as a real-time and probabilistic model which is implemented on the digital signal processor (DSP) computing device. To ensure the performance of the proposed approach and output behavior of the system, software-in-loop (SIL) and hardware-in-loop (HIL) testing procedures have been considered. From the simulation and implementation outcomes, it can be concluded that the proposed backstepping controller based DDPG is more effective, robust, and adaptive than the backstepping and proportional-integral (PI) controllers optimized by particle swarm optimization (PSO) in the presence of uncertainties and disturbances.

[1]  J. S. Lather,et al.  MODERN CONTROL ASPECTS INDOUBLY FED INDUCTION GENERATORBASED POWER SYSTEMS: A REVIEW , 2013 .

[2]  Shufeng Sun,et al.  Data-driven digital twin technology for optimized control in process systems. , 2019, ISA transactions.

[3]  Michael S. Selig,et al.  Application of a genetic algorithm to wind turbine design , 1996 .

[4]  Martin A. Riedmiller.  Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method , 2005, ECML.

[5]  Hali Pang,et al.  Deep Deterministic Policy Gradient for Traffic Signal Control of Single Intersection , 2019, 2019 Chinese Control And Decision Conference (CCDC).

[6]  Edward H. Glaessgen,et al.  The Digital Twin Paradigm for Future NASA and U.S. Air Force Vehicles , 2012 .

[7]  Marco Wiering,et al.  Ensemble Algorithms in Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  M. Khooban,et al.  DC/DC Power Converter Control-Based Deep Machine Learning Techniques: Real-Time Implementation , 2020, IEEE Transactions on Power Electronics.

[9]  Mohammad Hassan Khooban,et al.  Digital Twins-Assisted Design of Next-Generation Advanced Controllers for Power Systems and Electronics: Wind Turbine as a Case Study , 2020 .

[10]  Vahid Rezaei.  Advanced control of wind turbines: Brief survey, categorization, and challenges , 2015, 2015 American Control Conference (ACC).

[11]  Lin Jiang,et al.  Nonlinear PI control for variable pitch wind turbine , 2016 .

[12]  Zhe Zhang,et al.  Reinforcement-Learning-Based Intelligent Maximum Power Point Tracking Control for Wind Energy Conversion Systems , 2015, IEEE Transactions on Industrial Electronics.

[13]  Abdelhamid Rabhi,et al.  Robust Model-Free Control Applied to a Quadrotor UAV , 2016, J. Intell. Robotic Syst..

[14]  Saravanakumar Rajendran,et al.  Control of Variable Speed Variable Pitch Wind Turbine at Above and Below Rated Wind Speed , 2014 .

[15]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[16]  Houria Siguerdidjane,et al.  Comparison between linear and nonlinear control strategies for variable speed wind turbines , 2010 .

[17]  Jie Chen,et al.  New Overall Power Control Strategy for Variable-Speed Fixed-Pitch Wind Turbines Within the Whole Wind Velocity Range , 2013, IEEE Transactions on Industrial Electronics.

[18]  Abdelhamid Rabhi,et al.  Nonlinear Integral Backstepping ─ Model-Free Control Applied to a Quadrotor System , 2014 .

[19]  M.A. Wiering,et al.  Two Novel On-policy Reinforcement Learning Algorithms based on TD(λ)-methods , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[20]  Mohammad Hassan Khooban,et al.  A Novel Deep Reinforcement Learning Controller Based Type-II Fuzzy System: Frequency Regulation in Microgrids , 2021, IEEE Transactions on Emerging Topics in Computational Intelligence.

[21]  Wang Shaoping,et al.  High performance direct torque control of electrical aerodynamics load simulator using adaptive fuzzy backstepping control , 2015 .

[22]  Mohammad Hassan Khooban,et al.  An Intelligent Non-Integer PID Controller-Based Deep Reinforcement Learning: Implementation and Experimental Results , 2021, IEEE Transactions on Industrial Electronics.

[23]  Mohammad Hassan Khooban,et al.  Reliable Power Scheduling of an Emission-Free Ship: Multiobjective Deep Reinforcement Learning , 2020, IEEE Transactions on Transportation Electrification.

[24]  Sergey Levine,et al.  Continuous Deep Q-Learning with Model-based Acceleration , 2016, ICML.

[25]  William Leithead,et al.  Classical control of active pitch regulation of constant speed horizontal axis wind turbines , 1992 .

[26]  A. D. Wright,et al.  Modern Control Design for Flexible Wind Turbines , 2004 .

[27]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[28]  Houria Siguerdidjane,et al.  Nonlinear Control of a Variable-Speed Wind Turbine Using a Two-Mass Model , 2011, IEEE Transactions on Energy Conversion.

[29]  Andrew W. Moore,et al.  Prioritized sweeping: Reinforcement learning with less data and less time , 2004, Machine Learning.

[30]  Peter L. Bartlett,et al.  Infinite-Horizon Policy-Gradient Estimation , 2001, J. Artif. Intell. Res..

[31]  Saravanakumar RAJENDRAN,et al.  Backstepping sliding mode control of a variable speed wind turbine for power optimization , 2015 .

[32]  Andrew Y. C. Nee,et al.  Digital twin-driven product design framework , 2019, Int. J. Prod. Res..

[33]  Helge Aagaard Madsen,et al.  Optimization method for wind turbine rotors , 1999 .

[34]  Mohammad Hassan Khooban,et al.  A Novel Deep Learning Controller for DC–DC Buck–Boost Converters in Wireless Power Transfer Feeding CPLs , 2021, IEEE Transactions on Industrial Electronics.

[35]  Michael H. Bowling,et al.  Actor-Critic Policy Optimization in Partially Observable Multiagent Environments , 2018, NeurIPS.

[36]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[37]  Usama Hamayun Ghouri,et al.  Attitude Control of Quad-copter using Deterministic Policy Gradient Algorithms (DPGA) , 2019, 2019 2nd International Conference on Communication, Computing and Digital systems (C-CODE).

[38]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[39]  Cédric Join,et al.  Model-free control and intelligent PID controllers: towards a possible trivialization of nonlinear control? , 2009, ArXiv.

[40]  Hriday Bavle,et al.  A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform , 2018, Journal of Intelligent & Robotic Systems.

[41]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .