Q-Learning based Protection Scheme for Microgrid using Multi-Agent System

Distributed Energy Resources (DERs) such as Distributed Generators (DGs) or storage systems can be integrated with the central power distribution system. However, one of the severe challenges posed by the penetration of DGs to the utility grid system is the bi-directional power flows in the feeders. The bi-directional energy flows cause issues pertaining to the failures of protection systems because usually relays are designed to protect the network under unidirectional power flow case. Therefore, it is essential to have a robust protection scheme in support of distributed generators to protect the system from various faults. This paper proposes a novel protection scheme based on Q-learning and multi-agent system that identifies and isolates the different types of failure. The Q-learning algorithm is built to teach agents for making responsible decisions in fault identification and clearing. Furthermore, decentralized Blockchain based connections were adopted for exchanging information between agents. The system was simulated in MATLAB and JADE platforms and the results have shown that system is capable to identify different type of faults based on various states.

[1]  B. K. Panigrahi,et al.  Framework arrangement of directional relays in meshed networks based on differential evolution algorithm , 2016, 2016 IEEE 6th International Conference on Power Systems (ICPS).

[2]  Hiroshi Tsujino,et al.  Connectionist Reinforcement Learning with Cursory Intrinsic Motivations and Linear Dependencies to Multiple Representations , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[3]  Yih-Der Lee,et al.  Application of Multi-agent Systems to Microgrid Fault Protection Coordination , 2016, 2016 International Symposium on Computer, Consumer and Control (IS3C).

[4]  Wesley P. Calixto,et al.  Optimized Coordination of Directional Overcurrent Relays Using Genetic Algorithm , 2018, 2018 IEEE International Conference on Environment and Electrical Engineering and 2018 IEEE Industrial and Commercial Power Systems Europe (EEEIC / I&CPS Europe).

[5]  Suryanarayana Doolla,et al.  Decentralized transactive energy management system for distribution systems with prosumer microgrids , 2018, 2018 19th International Carpathian Control Conference (ICCC).

[6]  Osama Mohammed,et al.  Decentralized Multi-Agent System for Protection and the Power Restoration Process in Microgrids , 2017, 2017 Ninth Annual IEEE Green Technologies Conference (GreenTech).

[7]  J. Borroy,et al.  Current phase comparison pilot scheme for distributed generation networks protection , 2011 .

[8]  Sanyam Kapoor,et al.  Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches , 2018, ArXiv.

[9]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[10]  Karin Hinzer,et al.  High-Reliability Multi-Agent Q-Learning-Based Scheduling for D2D Microgrid Communications , 2019, IEEE Access.

[11]  Mahmoud-Reza Haghifam,et al.  Load management using multi-agent systems in smart distribution network , 2013, 2013 IEEE Power & Energy Society General Meeting.

[12]  Haili Song,et al.  Optimal electricity supply bidding by Markov decision process , 2000 .

[13]  Qi Liu,et al.  Multi-agent based protection system for distribution system with DG , 2013, ArXiv.

[14]  Wesley P. Calixto,et al.  Optimized setting of directional overcurrent relays via genetic algorithm , 2018, 2018 Simposio Brasileiro de Sistemas Eletricos (SBSE).

[15]  Jacqueline G. Rolim,et al.  Multi-agent system for adaptive protection in microgrids , 2013, 2013 IEEE PES Conference on Innovative Smart Grid Technologies (ISGT Latin America).

[16]  Il-Woo Lee,et al.  Distributed energy resources monitoring and control based on IoT protocol , 2017, 2017 International Conference on Information and Communication Technology Convergence (ICTC).

[17]  Josep M. Guerrero,et al.  Optimal Overcurrent Relay Coordination in Presence of Inverter-Based Wind Farms and Electrical Energy Storage Devices , 2018, 2018 IEEE International Conference on Environment and Electrical Engineering and 2018 IEEE Industrial and Commercial Power Systems Europe (EEEIC / I&CPS Europe).

[18]  Semih C Ilik,et al.  Effects of Distributed Generation on Overcurrent Relay Coordination and an Adaptive Protection Scheme , 2017 .

[19]  Gaetano Zizzo,et al.  A Technical Approach to the Energy Blockchain in Microgrids , 2018, IEEE Transactions on Industrial Informatics.

[20]  Sijie Chen,et al.  Smart contract-based campus demonstration of decentralized transactive energy auctions , 2017, 2017 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT).