A Hybrid Multiagent Framework With Q-Learning for Power Grid Systems Restoration

This paper presents a hybrid multiagent framework with a Q-learning algorithm to support rapid restoration of power grid systems following catastrophic disturbances involving loss of generators. This framework integrates the advantages of both centralized and decentralized architectures to achieve accurate decision making and quick responses when potential cascading failures are detected in power systems. By using this hybrid framework, which does not rely on a centralized controller, the single point of failure in power grid systems can be avoided. Further, the use of the Q-learning algorithm developed in conjunction with the restorative framework can help the agents to make accurate decisions to protect against cascading failures in a timely manner without requiring a global reward signal. Simulation results demonstrate the effectiveness of the proposed approach in comparison with the typical centralized and decentralized approaches based on several evaluation attributes.

[1]  James S. Thorp,et al.  Agent technology applied to adaptive relay setting for multi-terminal lines , 2000, 2000 Power Engineering Society Summer Meeting (Cat. No.00CH37134).

[2]  Michael Winikoff,et al.  Debugging multi-agent systems using design artifacts: the case of interaction protocols , 2002, AAMAS '02.

[3]  M. Lehtonen,et al.  Distributed agent-based State estimation for electrical distribution networks , 2005, IEEE Transactions on Power Systems.

[4]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[5]  Ryszard Kowalczyk,et al.  Dynamic analysis of multiagent Q-learning with ε-greedy exploration , 2009, ICML '09.

[6]  C. C. Liu,et al.  Adaptation in Load Shedding under Vulnerable Operation Conditions , 2002, IEEE Power Engineering Review.

[7]  H. Sasaki,et al.  A multi-agent approach to power system restoration , 2000, PowerCon 2000. 2000 International Conference on Power System Technology. Proceedings (Cat. No.00EX409).

[8]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[9]  Kai Huang,et al.  Intelligent Agents Applied to Reconfiguration of Mesh Structured Power Systems , 2007, 2007 International Conference on Intelligent Systems Applications to Power Systems.

[10]  Victor R. Lesser,et al.  Integrating organizational control into multi-agent learning , 2009, AAMAS.

[11]  Q. H. Wu,et al.  Agent-based substation automation , 2003 .

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  N.N. Schulz,et al.  A Multi-Agent Solution to Distribution Systems Restoration , 2007, IEEE Transactions on Power Systems.

[14]  T. Nagata,et al.  Decentralized approach to normal operations for power system network , 2005, Proceedings of the 13th International Conference on, Intelligent Systems Application to Power Systems.

[15]  Shie Mannor,et al.  PAC Bounds for Multi-armed Bandit and Markov Decision Processes , 2002, COLT.

[16]  Karen L. Butler-Purry,et al.  Multi-Agent System for Automated Service Restoration of Shipboard Power Systems , 2002, CAINE.

[17]  Tommi S. Jaakkola,et al.  Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms , 2000, Machine Learning.

[18]  Chen-Ching Liu,et al.  Multi-agent technology for vulnerability assessment and control , 2001, 2001 Power Engineering Society Summer Meeting. Conference Proceedings (Cat. No.01CH37262).

[19]  X. Xu,et al.  Improving circuit breaker maintenance management tasks by applying mobile agent software technology , 2002, IEEE/PES Transmission and Distribution Conference and Exhibition.

[20]  J.A. Momoh,et al.  Optimal Reconfiguration of the Navy Ship Power System using Agents , 2006, 2005/2006 IEEE/PES Transmission and Distribution Conference and Exhibition.