Multi-machine power system control based on dual heuristic dynamic programming

In this paper, we integrate a goal network into the existing dual heuristic dynamic programming (DHP) architecture, and study its damping performance on the multi-machine power system. There are four types of neural network in our proposed design: a goal network, a critic network, an action network and a model network. The motivation of this design is to build a general mapping between the system variables and the partial derivatives of the utility function, so that these required derivatives can be directly obtained and adaptively tuned over time. However, the existing DHP design can only obtain a predefined (fixed) external utility function (or its derivatives). We apply both the proposed approach and the existing DHP approach on the multi-machine power system, and compare the damping performance on a four-machine two-area power system. The simulation results demonstrate the improved control performance with the proposed design.

[1]  Haibo He,et al.  Power System Stability Control for a Wind Farm Based on Adaptive Dynamic Programming , 2015, IEEE Transactions on Smart Grid.

[2]  Haibo He,et al.  A three-network architecture for on-line learning and optimization based on adaptive dynamic programming , 2012, Neurocomputing.

[3]  Jinyu Wen,et al.  Energy-Storage-Based Low-Frequency Oscillation Damping Control Using Particle Swarm Optimization and Heuristic Dynamic Programming , 2014, IEEE Transactions on Power Systems.

[4]  Haibo He,et al.  Real-time tracking on adaptive critic design with uniformly ultimately bounded condition , 2013, 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[5]  Paul J. Werbos,et al.  Computational Intelligence for the Smart Grid-History, Challenges, and Opportunities , 2011, IEEE Computational Intelligence Magazine.

[6]  R. Harley,et al.  Computational Intelligence in Smart Grids , 2011 .

[7]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[8]  Babu Narayanan,et al.  POWER SYSTEM STABILITY AND CONTROL , 2015 .

[9]  Haibo He,et al.  GrDHP: A General Utility Function Representation for Dual Heuristic Dynamic Programming , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[10]  R.G. Harley,et al.  Adaptive Critic Design Based Neuro-Fuzzy Controller for a Static Compensator in a Multimachine Power System , 2006, 2007 IEEE Power Engineering Society General Meeting.

[11]  Ieee Staff.  2014 IEEE Symposium on Computational Intelligence Applications in Smart Grid (CIASG) , 2014 .

[12]  Jinyu Wen,et al.  Adaptive Learning in Tracking Control Based on the Dual Critic Network Design , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Ganesh Kumar Venayagamoorthy,et al.  Two-Level Dynamic Stochastic Optimal Power Flow Control for Power Systems With Intermittent Renewable Generation , 2014, IEEE Transactions on Power Systems.

[14]  Haibo He,et al.  Adaptive control for an HVDC transmission link with FACTS and a wind farm , 2013, 2013 IEEE PES Innovative Smart Grid Technologies Conference (ISGT).

[15]  Haibo He,et al.  Optimal Control for Unknown Discrete-Time Nonlinear Markov Jump Systems Using Adaptive Dynamic Programming , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Haibo He,et al.  Frequency control using on-line learning method for island smart grid with EVs and PVs , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[17]  Haibo He,et al.  Goal Representation Heuristic Dynamic Programming on Maze Navigation , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Chao Lu,et al.  Direct Heuristic Dynamic Programming for Damping Oscillations in a Large Power System , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Haibo He,et al.  Heuristic dynamic programming with internal goal representation , 2013, Soft Comput..

[20]  Haibo He.  Self-Adaptive Systems for Machine Intelligence , 2011 .

[21]  G. Venayagamoorthy,et al.  Two separate continually online trained neurocontrollers for excitation and turbine control of a turbogenerator , 2000, Conference Record of the 2000 IEEE Industry Applications Conference. Thirty-Fifth IAS Annual Meeting and World Conference on Industrial Applications of Electrical Energy (Cat. No.00CH37129).

[22]  Haibo He,et al.  Comparative study between HDP and PSS on DFIG damping control , 2013, 2013 IEEE Computational Intelligence Applications in Smart Grid (CIASG).

[23]  Haibo He.  Self-Adaptive Systems for Machine Intelligence: He/Machine Intelligence , 2011 .

[24]  Haibo He,et al.  Reactive power control of grid-connected wind farm based on adaptive dynamic programming , 2014, Neurocomputing.

[25]  Haibo He,et al.  Reinforcement learning control based on multi-goal representation using hierarchical heuristic dynamic programming , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[26]  Frank L. Lewis,et al.  Reinforcement Learning and Approximate Dynamic Programming for Feedback Control , 2012 .