Evaluation of Reinforcement Learning-Based False Data Injection Attack to Automatic Voltage Control

False data injection (FDI) attacks intend to threaten the security of power systems. In this paper, a novel strategy of FDI attacks is proposed, which aims to distort normal operation of a power system regulated by automatic voltage controls (AVCs). Such attacks can be launched from a single substation by the attacker who has little knowledge of the whole power grid. The optimal attack strategy is modeled as a partial observable Markov decision process (POMDP). Then, a $\mathcal {Q}$ -learning algorithm with nearest sequence memory is adopted to enable on-line learning and attacking. Stealthy attack strategies are also developed and incorporated into the POMDP model. Various tests are performed upon the IEEE 39-bus systems. Corresponding results verify the efficacy of the proposed attack strategies. The feasibility of independent and data-driven FDI attacks is investigated. Moreover, a bad data detection and correction method are presented based on kernel density estimation to mitigate the disruptive impacts of the proposed FDI attacks. Test results show that this defensive method can help maintain the security of the AVC system, even under heavy system loading.

[1]  G. Manimaran,et al.  Vulnerability Assessment of Cybersecurity for SCADA Systems , 2008, IEEE Transactions on Power Systems.

[2]  Haibo He,et al.  Cyber-physical attacks and defences in the smart grid: a survey , 2016, IET Cyper-Phys. Syst.: Theory & Appl..

[3]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[4]  L. Tolbert,et al.  Review of reactive power planning: objectives, constraints, and algorithms , 2008, T&D 2008.

[5]  Gabriela Hug,et al.  Vulnerability Assessment of AC State Estimation With Respect to False Data Injection Cyber-Attacks , 2012, IEEE Transactions on Smart Grid.

[6]  Aditya Ashok,et al.  Cyber-Physical Security Testbeds: Architecture, Application, and Evaluation for Smart Grid , 2013, IEEE Transactions on Smart Grid.

[7]  Bart De Schutter,et al.  Reinforcement Learning and Dynamic Programming Using Function Approximators , 2010 .

[8]  Zuyi Li,et al.  Modeling Load Redistribution Attacks in Power Systems , 2011, IEEE Transactions on Smart Grid.

[9]  Zuyi Li,et al.  Quantitative Analysis of Load Redistribution Attacks in Power Systems , 2012, IEEE Transactions on Parallel and Distributed Systems.

[10]  G. Manimaran,et al.  Data integrity attack and its impacts on voltage control loop in power grid , 2011, 2011 IEEE Power and Energy Society General Meeting.

[11]  Peng Ning,et al.  False data injection attacks against state estimation in electric power grids , 2011, TSEC.

[12]  A. G. Expósito,et al.  Power system state estimation : theory and implementation , 2004 .

[13]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[14]  Zuyi Li,et al.  Local Load Redistribution Attacks in Power Systems With Incomplete Network Information , 2014, IEEE Transactions on Smart Grid.

[15]  Jinping Hao,et al.  Sparse Malicious False Data Injection Attacks and Defense Mechanisms in Smart Grids , 2015, IEEE Transactions on Industrial Informatics.

[16]  Adnan Anwar,et al.  Data-Driven Stealthy Injection Attacks on Smart Grid with Incomplete Measurements , 2016, PAISI.

[17]  Lang Tong,et al.  Data Framing Attack on State Estimation , 2013, IEEE Journal on Selected Areas in Communications.

[18]  Heejo Lee,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. INVITED PAPER Cyber–Physical Security of a Smart Grid Infrastructure , 2022 .

[19]  Haibo He,et al.  Q-Learning-Based Vulnerability Analysis of Smart Grid Against Sequential Topology Attacks , 2017, IEEE Transactions on Information Forensics and Security.

[20]  Ying Chen,et al.  Cyber security assessment of wide area controlled power system based on co-simulations , 2014, 2014 International Conference on Power System Technology.

[21]  Wen-Long Chin,et al.  Blind False Data Injection Attack Using PCA Approximation Method in Smart Grid , 2015, IEEE Transactions on Smart Grid.

[22]  Stuxnet: targeting Iran's nuclear programme , 2011 .

[23]  Lang Tong,et al.  Subspace Methods for Data Attack on State Estimation: A Data Driven Approach , 2014, IEEE Transactions on Signal Processing.

[24]  Siddharth Sridhar,et al.  Model-Based Attack Detection and Mitigation for Automatic Generation Control , 2014, IEEE Transactions on Smart Grid.

[25]  Bruno Sinopoli,et al.  Integrity Data Attacks in Power Market Operations , 2011, IEEE Transactions on Smart Grid.