SAVER: Safe Learning-Based Controller for Real-Time Voltage Regulation

Fast and safe voltage regulation algorithms can serve as fundamental schemes for achieving a high level of renewable penetration in the modern distribution power grids. Faced with uncertain or even unknown distribution grid models and fastchanging power injections, model-free deep reinforcement learning (DRL) algorithms have been proposed to find the reactive power injections for inverters while optimizing the voltage profiles. However, such data-driven controllers can not guarantee satisfaction of the hard operational constraints, such as maintaining voltage profiles within a certain range of the nominal value. To this end, we propose SAVER: SAfe VoltagE Regulator, which is composed of an RL learner and a specifically designed, computational efficient safety projection layer. SAVER provides a plug-and-play interface for a set of DRL algorithms that guarantees the system voltages to be within safe bounds. Numerical simulations on real-world data validate the performance of the proposed algorithm.

[1]  Na Li,et al.  Optimal Distributed Feedback Voltage Control Under Limited Reactive Power , 2018, IEEE Transactions on Power Systems.

[2]  Hao Zhu,et al.  Data-driven Modeling for Distribution Grids Under Partial Observability , 2021, 2021 North American Power Symposium (NAPS).

[3]  Lin Gao,et al.  Safe deep reinforcement learning-based constrained optimal control scheme for active distribution networks , 2020 .

[4]  K. Mani Chandy,et al.  Inverter VAR control for distribution systems with renewables , 2011, 2011 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[5]  Michael Chertkov,et al.  Structure Learning in Power Distribution Networks , 2015, IEEE Transactions on Control of Network Systems.

[6]  Haotian Liu,et al.  Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control , 2020, ArXiv.

[7]  Baosen Zhang,et al.  Input Convex Neural Networks for Optimal Voltage Regulation , 2020, 2002.08684.

[8]  M. E. Baran,et al.  Optimal capacitor placement on radial distribution systems , 1989 .

[9]  T. L. Vandoorn,et al.  Voltage-Based Droop Control of Renewables to Avoid On–Off Oscillations Caused by Overvoltages , 2013, IEEE Transactions on Power Delivery.

[10]  Sean Peisert,et al.  Understanding the Safety Requirements for Learning-based Power Systems Operations , 2021, ArXiv.

[11]  Zhenhuan Ding,et al.  A Guided Deep Reinforcement Learning Method For Distribution Voltage Regulation via Battery Systems , 2021, 2021 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT).

[12]  Michael Chertkov,et al.  Exact Topology and Parameter Estimation in Distribution Grids with Minimal Observability , 2017, 2018 Power Systems Computation Conference (PSCC).

[13]  L.A.F. Ferreira,et al.  Distributed Reactive Power Generation Control for Voltage Rise Mitigation in Distribution Networks , 2008, IEEE Transactions on Power Systems.

[14]  Hanchen Xu,et al.  Optimal Tap Setting of Voltage Regulation Transformers Using Batch Reinforcement Learning , 2018, IEEE Transactions on Power Systems.

[15]  Yinyu Ye,et al.  An extension of Karmarkar's projective algorithm for convex quadratic programming , 1989, Math. Program..

[16]  Baosen Zhang,et al.  Learning to Solve Network Flow Problems via Neural Decoding , 2020, 2002.04091.

[17]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[18]  Munther A. Dahleh,et al.  Real-time decentralized voltage control in distribution networks , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[19]  Yuval Tassa,et al.  Safe Exploration in Continuous Action Spaces , 2018, ArXiv.

[20]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[21]  Wei Wang,et al.  Safe Off-Policy Deep Reinforcement Learning Algorithm for Volt-VAR Control in Power Distribution Systems , 2020, IEEE Transactions on Smart Grid.