Distributed multi-step Q(λ) learning for Optimal Power Flow of large-scale power grids

Abstract This paper presents a novel distributed multi-step Q(λ) learning algorithm (DQ(λ)L) based on multi-agent system for solving large-scale multi-objective OPF problem. It does not require any manipulation to the conventional mathematical Optimal Power Flow (OPF) model. Large-scale power system is first partitioned to subsystems and each subsystem is managed by an agent. Each agent adopts the standard multi-step Q(λ) learning algorithm to pursue its own objectives independently and approaches to the global optimal through cooperation and coordination among agents. The proposed DQ(λ)L has been thoroughly studied and tested on the IEEE 9-bus and 118-bus systems. Case studies demonstrated that DQ(λ)L is a feasible and effective for solving multi-objective OPF problem in large-scale complex power grid.

[1]  R. Jabr,et al.  A Primal-Dual Interior Point Method for Optimal Power Flow Dispatching , 2002, IEEE Power Engineering Review.

[2]  Gerhard Weiß,et al.  Distributed reinforcement learning , 1995, Robotics Auton. Syst..

[3]  Jiexin Pu,et al.  Distributed Multi-agent Reinforcement Learning and Its Application to Robot Soccer , 2008, 2008 International Workshop on Education Technology and Training & 2008 International Workshop on Geoscience and Remote Sensing.

[4]  Abhinav Sadu,et al.  A hybrid multi-agent based particle swarm optimization algorithm for economic power dispatch , 2011 .

[5]  N.D. Hatziargyriou,et al.  Reinforcement learning for reactive power control , 2004, IEEE Transactions on Power Systems.

[6]  Luonan Chen,et al.  Mean field theory for optimal power flow , 1997 .

[7]  K.L. Lo,et al.  Power loss allocation in pool-based electricity markets , 2008, 2008 Australasian Universities Power Engineering Conference.

[8]  B. H. Kim,et al.  A comparison of distributed optimal power flow algorithms , 2000 .

[9]  Mohamed A. El-Sharkawi,et al.  Modern heuristic optimization techniques :: theory and applications to power systems , 2008 .

[10]  A. Renaud,et al.  Daily generation scheduling optimization with transmission constraints: a new class of algorithms , 1992 .

[11]  Hsiao-Dong Chiang,et al.  Power-current hybrid rectangular formulation for interior-point optimal power flow , 2009 .

[12]  Aggelos S. Bouhouras,et al.  Cost/worth assessment of reliability improvement in distribution networks by means of artificial intelligence , 2010 .

[13]  M.E.H. Benbouzid,et al.  Optimal power flow for large-scale power system with shunt FACTS using efficient parallel GA , 2008, 2008 34th Annual Conference of IEEE Industrial Electronics.

[14]  Wanxing Sheng,et al.  The decomposition and computation method for distributed optimal power flow based on message passing interface (MPI) , 2011 .

[15]  Gehao Sheng,et al.  Framework and implementation of secondary voltage regulation strategy based on multi-agent technology , 2009 .

[16]  S. Y. Wang,et al.  A hybrid decoupled approach to optimal power flow , 1996 .

[17]  Damien Ernst,et al.  A comparison of Nash equilibria analysis and agent-based modelling for power markets , 2006 .

[18]  C. Jiang,et al.  Improved evolutionary programming with dynamic mutation and metropolis criteria for multi-objective reactive power optimisation , 2005 .

[19]  Tao Yu,et al.  Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step $Q(\lambda)$ Learning , 2011, IEEE Transactions on Power Systems.

[20]  Ross Baldick,et al.  Coarse-grained distributed optimal power flow , 1997 .

[21]  T. Bouktir,et al.  Optimal power flow for large-scale power system with shunt FACTS using fast parallel GA , 2008, MELECON 2008 - The 14th IEEE Mediterranean Electrotechnical Conference.

[22]  E. A. Jasmin,et al.  Reinforcement Learning approaches to Economic Dispatch problem , 2011 .

[23]  Mario Russo,et al.  On the Application of the Auxiliary Problem Principle , 2003 .