Near optimal control policy for controlling power system stabilizers using reinforcement learning

In this paper, a reinforcement learning method called Q-learning is applied to find a near optimal control policy for controlling power system stabilizers (PSS). The single agent approach is used, but the design procedure can be expanded to a multi-agent system. The objective of the control policy is to enhance the stability of a multi-machine power system by increasing the damping ratio of the least damped modes. By learning a near optimal policy, not only the design parameters of the PSSs are simultaneously optimized, but also the agents can track the system changes and update the parameters of PSSs. The off-line mode of operation is used in this paper after achieving the near optimal policy. The validity of proposed method has been tested on a 2 area, 4 machines power system.

[1]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[2]  M. J. Gibbard,et al.  Reconciliation of methods of compensation for PSSs in multimachine systems , 2004 .

[3]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[4]  A. Feliachi,et al.  Stability enhancement through reinforcement learning: Load frequency control case study , 2007, 2007 iREP Symposium - Bulk Power System Dynamics and Control - VII. Revitalizing Operational Reliability.

[5]  Mevludin Glavic,et al.  Design of a resistive brake controller for power system stability enhancement using reinforcement learning , 2005, IEEE Transactions on Control Systems Technology.

[6]  E. Larsen,et al.  IEEE Transactions on Power Apparatus and Systems, Vol. PAS-100, No. 6 June 1981 APPLYING POWER SYSTEM STABILIZERS PART I: GENERAL CONCEPTS , 2006 .

[7]  M. A. Abido,et al.  Optimal Design of Power System Stabilizers Using Evolutionary Programming , 2002, IEEE Power Engineering Review.

[8]  M. A. Abido,et al.  Simultaneous stabilization of multimachine power systems via genetic algorithms , 1999, IEEE Transactions on Power Systems.

[9]  D. Ernst,et al.  Power systems stability control: reinforcement learning framework , 2004, IEEE Transactions on Power Systems.

[10]  N.D. Hatziargyriou,et al.  Reinforcement learning for reactive power control , 2004, IEEE Transactions on Power Systems.

[11]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[12]  I. Erlich,et al.  Simultaneous coordinated tuning of PSS and FACTS damping controllers in large power systems , 2005, IEEE Transactions on Power Systems.

[13]  Graham Rogers,et al.  Power System Oscillations , 1999 .