Reinforcement Learning based Proactive Control for Transmission Grid Resilience to Wildfire

Power grid operation subject to an extreme event requires decision-making by human operators under stressful condition with high cognitive load. Decision support under adverse dynamic events, specially if forecasted, can be supplemented by intelligent proactive control. Power system operation during wildfires require resiliency-driven proactive control for load shedding, line switching and resource allocation considering the dynamics of the wildfire and failure propagation. However, possible number of lineand load-switching in a large system during an event make traditional prediction-driven and stochastic approaches computationally intractable, leading operators to often use greedy algorithms. We model and solve the proactive control problem as a Markov decision process and introduce an integrated testbed for spatio-temporal wildfire propagation and proactive power-system operation. We transform the enormous wildfire-propagation observation space and utilize it as part of a heuristic for proactive de-energization of transmission assets. We integrate this heuristic with a reinforcement-learning based proactive policy for controlling the generating assets. Our approach allows this controller to provide setpoints for a part of the generation fleet, while a myopic operator can determine the setpoints for the remaining set, which results in a symbiotic action. We evaluate our approach utilizing the IEEE 24-node system mapped on a hypothetical terrain. Our results show that the proposed approach can help the operator to reduce load loss during an extreme event, reduce power flow through lines that are to be de-energized, and reduce the likelihood of infeasible power-flow solutions, which would indicate violation of shortterm thermal limits of transmission lines.

[1]  S. Low,et al.  Reinforcement Learning for Decision-Making and Control in Power Systems: Tutorial, Review, and Vision , 2021, ArXiv.

[2]  Line A. Roald,et al.  Balancing Wildfire Risk and Power Outages Through Optimized Power Shut-Offs , 2020, IEEE Transactions on Power Systems.

[3]  J. Abatzoglou,et al.  Population exposure to pre-emptive de-energization aimed at averting wildfires in Northern California , 2020, Environmental Research Letters.

[4]  Nikos D. Hatziargyriou,et al.  Robust Resiliency-Oriented Operation of Active Distribution Networks Considering Windstorms , 2020, IEEE Transactions on Power Systems.

[5]  Shuchismita Biswas,et al.  Proactive Islanding of the Power Grid to Mitigate High-Impact Low-Frequency Events , 2020, 2020 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT).

[6]  Anamika Dubey,et al.  Critical Load Restoration Using Distributed Energy Resources for Resilient Power Distribution System , 2019, IEEE Transactions on Power Systems.

[7]  Tao Jin,et al.  Proactive Resilience of Power Systems Against Natural Disasters: A Literature Review , 2019, IEEE Access.

[8]  Yin Xu,et al.  Enabling Resiliency Operations Across Multiple Microgrids With Grid Friendly Appliance Controllers , 2018, IEEE Transactions on Smart Grid.

[9]  Munther A. Dahleh,et al.  Advancing systems and control research in the era of ML and AI , 2018, Annu. Rev. Control..

[10]  Mykel J. Kochenderfer,et al.  A comparison of Monte Carlo tree search and rolling horizon optimization for large-scale dynamic resource allocation problems , 2017, Eur. J. Oper. Res..

[11]  Payman Dehghanian,et al.  Quantifying power system resiliency improvement using network reconfiguration , 2017, 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS).

[12]  Chong Wang,et al.  Resilience Enhancement With Sequentially Proactive Operation Strategies , 2017, IEEE Transactions on Power Systems.

[13]  H. Vincent Poor,et al.  Resilience of Energy Infrastructure and Services: Modeling, Data Analytics, and Metrics , 2016, Proceedings of the IEEE.

[14]  Pierluigi Mancarella,et al.  Boosting the Power Grid Resilience to Extreme Weather Events Using Defensive Islanding , 2016, IEEE Transactions on Smart Grid.

[15]  Ross Baldick,et al.  Research on Resilience of Power Systems Under Natural Disasters—A Review , 2016, IEEE Transactions on Power Systems.

[16]  Jianhui Wang,et al.  Resilient Distribution System by Microgrids Formation After Natural Disasters , 2016, IEEE Transactions on Smart Grid.

[17]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[18]  Weiyi Chen,et al.  Whether the High-Voltage Transmission Lines Have Enough Load Capacity After Wildfire , 2016 .

[19]  Salman Mohagheghi,et al.  Vulnerability assessment of the power grid against progressing wildfires , 2015 .

[20]  Seth D. Guikema,et al.  Predicting Hurricane Power Outages to Support Storm Response Planning , 2014, IEEE Access.

[21]  Seth D. Guikema,et al.  Forecasting hurricane-induced power outage durations , 2014, Natural Hazards.

[22]  R. Katz,et al.  US billion-dollar weather and climate disasters: data sources, trends, accuracy and biases , 2013, Natural Hazards.

[23]  V. Terzija,et al.  Controlled islanding strategy considering power system restoration constraints , 2012, 2012 IEEE Power and Energy Society General Meeting.

[24]  David A. Stanford,et al.  A stochastic forest fire growth model , 2009, Environmental and Ecological Statistics.

[25]  Jery R. Stedinger,et al.  Negative Binomial Regression of Electric Power Outages in Hurricanes , 2005 .

[26]  Goran Strbac,et al.  System Security and Ancillary Services , 2005 .

[27]  W. Gorzegno,et al.  Load Rejection Capability for Large Steam Generators , 1983, IEEE Power Engineering Review.