Coordinated Topology Attacks in Smart Grid Using Deep Reinforcement Learning

In this article, we investigate the coordinated topology attacks in smart grid, which combine a physical topology attack and a cyber-topology attack. The physical attack first trips a transmission line. In order to deceive the control center, the attacker masks the outage signal of the tripped line in the cyber layer and then creates a fake outage signal for another transmission line. The goal of coordinated topology attacks is to overload a critical line (different from the physical tripped line and the fake outage line) by misleading the control center into making improper dispatch. In order to determine the attack strategy, we propose a deep-reinforcement-learning-based method to identify the physical tripped line and the fake outage line. Besides, in order to block the outage signal of the tripped line and create the fake outage signal with limited attack resources, we propose a deep-reinforcement-learning-based approach to determine the minimal attack resources. Numerical simulations verify the effectiveness of the proposed method.

[1]  Gabriela Hug,et al.  Vulnerability Assessment of AC State Estimation With Respect to False Data Injection Cyber-Attacks , 2012, IEEE Transactions on Smart Grid.

[2]  Qi Wang,et al.  A two-layer game theoretical attack-defense model for a false data injection attack against power systems , 2019, International Journal of Electrical Power & Energy Systems.

[3]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[4]  Sherali Zeadally,et al.  Physical Layer Security for the Smart Grid: Vulnerabilities, Threats, and Countermeasures , 2019, IEEE Transactions on Industrial Informatics.

[5]  Lalitha Sankar,et al.  Implementation of unobservable state-preserving topology attacks , 2015, 2015 North American Power Symposium (NAPS).

[6]  Qinglai Wei,et al.  A reinforcement learning approach for sequential decision-making process of attacks in smart grid , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[7]  Zibin Zheng,et al.  Sequential Node/Link Recovery Strategy of Power Grids Based on Q-Learning Approach , 2019, 2019 IEEE International Symposium on Circuits and Systems (ISCAS).

[8]  Haibo He,et al.  Model-Free Real-Time EV Charging Scheduling Based on Deep Reinforcement Learning , 2019, IEEE Transactions on Smart Grid.

[9]  Lang Tong,et al.  On Topology Attack of a Smart Grid: Undetectable Attacks and Countermeasures , 2013, IEEE Journal on Selected Areas in Communications.

[10]  Zuyi Li,et al.  Cyber Cascades Screening Considering the Impacts of False Data Injection Attacks , 2018, IEEE Transactions on Power Systems.

[11]  Claudio R. Fuerte-Esquivel,et al.  A Sensitivity-Based Approach for the Detection of Multiple-Line Outages Using Phasor Measurements , 2019, IEEE Transactions on Power Systems.

[12]  Abdullah Abusorrah,et al.  Analyzing Locally Coordinated Cyber-Physical Attacks for Undetectable Line Outages , 2018, IEEE Transactions on Smart Grid.

[13]  Chau Yuen,et al.  Local Cyber-Physical Attack for Masking Line Outage and Topology Attack in Smart Grid , 2018, IEEE Transactions on Smart Grid.

[14]  Bri-Mathias Hodge,et al.  An Extended IEEE 118-Bus Test System With High Renewable Penetration , 2018, IEEE Transactions on Power Systems.

[15]  Zuyi Li,et al.  Modeling Load Redistribution Attacks in Power Systems , 2011, IEEE Transactions on Smart Grid.

[16]  Eklas Hossain,et al.  Application of Big Data and Machine Learning in Smart Grid, and Associated Security Concerns: A Review , 2019, IEEE Access.

[17]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[18]  Xiaohui Liang,et al.  Securing smart grid: cyber attacks, countermeasures, and challenges , 2012, IEEE Communications Magazine.

[19]  R D Zimmerman,et al.  MATPOWER: Steady-State Operations, Planning, and Analysis Tools for Power Systems Research and Education , 2011, IEEE Transactions on Power Systems.

[20]  Fangxing Li,et al.  Probabilistic LMP Forecasting Considering Load Uncertainty , 2009, IEEE Transactions on Power Systems.

[21]  Hao Liang,et al.  CCPA: Coordinated Cyber-Physical Attacks and Countermeasures in Smart Grid , 2017, IEEE Transactions on Smart Grid.

[22]  Lalitha Sankar,et al.  Physical System Consequences of Unobservable State-and-Topology Cyber-Physical Attacks , 2016, IEEE Transactions on Smart Grid.

[23]  T.J. Overbye,et al.  Line Outage Detection Using Phasor Angle Measurements , 2008, IEEE Transactions on Power Systems.

[24]  A. Ashok,et al.  Cyber attacks on power system state estimation through topology errors , 2012, 2012 IEEE Power and Energy Society General Meeting.

[25]  Abdullah Abusorrah,et al.  Bilevel Model for Analyzing Coordinated Cyber-Physical Attacks on Power Systems , 2016, IEEE Transactions on Smart Grid.

[26]  Ravishankar K. Iyer,et al.  Self-Healing Attack-Resilient PMU Network for Power System Operation , 2018, IEEE Transactions on Smart Grid.

[27]  Zuyi Li,et al.  Masking Transmission Line Outages via False Data Injection Attacks , 2016, IEEE Transactions on Information Forensics and Security.

[28]  Haibo He,et al.  Q-Learning-Based Vulnerability Analysis of Smart Grid Against Sequential Topology Attacks , 2017, IEEE Transactions on Information Forensics and Security.