Weighted interdependent network disintegration strategy based on Q-learning

Abstract The problem of network disintegration is one of the core topics in the field of network science. Currently, most of the existing research is based on homogeneous and single-layer networks of nodes. However, the various components of complex systems in the real world are often interdependent, and the cost of attacking different units is diverse, causing the traditional network disintegration method to lack good applicability. This paper establishes a weighted interdependent network (WIN) model, and based on this, a WIN disintegration strategy based on Q-learning is proposed. First, the network nodes are divided into multiple node sets according to the dependencies between the nodes, and the state and action space of Q-learning are determined. Next, the disintegration cost constraints and Q-learning parameters are defined to perform iterative learning. Then, the optimal network disintegration strategy is calculated according to the iterative Q-table. The results show that when the cost sensitivity factor ( p ) is fixed, DSQ can maintain good results in disintegrating different types of networks under different cost constraints, while the baseline methods have difficulty guaranteeing the disintegration effect in the face of different types of networks. Furthermore, we perform a sensitivity analysis on the p value and find that the effect of most of the baseline methods worsens as the p value increases, while DSQ maintains a good effect.

[1]  Rui Peng,et al.  Defender–Attacker Games with Asymmetric Player Utilities , 2019, Risk analysis : an official publication of the Society for Risk Analysis.

[2]  Dinesh C. Verma,et al.  Implicit Terrorist Networks: A Two-Mode Social Network Analysis of Terrorism in India , 2018, SBP-BRiMS.

[3]  Yamir Moreno,et al.  Focus on multilayer networks , 2019, New Journal of Physics.

[4]  Alessandro Vespignani,et al.  Twenty years of network science , 2018, Nature.

[5]  Harry Eugene Stanley,et al.  Robustness of interdependent networks under targeted attack , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  Jiang Jiang,et al.  Disintegration of Operational Capability of Heterogeneous Combat Networks Under Incomplete Information , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[7]  Yingwu Chen,et al.  Research on Functional Robustness of Heterogeneous Combat Networks , 2019, IEEE Systems Journal.

[8]  Xiaoming Xiong,et al.  The Action Control Model for Robotic Fish Using Improved Extreme Learning Machine , 2019, Complex..

[9]  R. Kulkarni,et al.  Corrigendum: IRS1 deficiency protects β-cells against ER stress-induced apoptosis by modulating sXBP-1 stability and protein translation , 2016, Scientific Reports.

[10]  Jinming Ma,et al.  Information Spreading on Weighted Multiplex Social Network , 2019, Complex..

[11]  Dawei Zhao,et al.  Virus Propagation and Patch Distribution in Multiplex Networks: Modeling, Analysis, and Optimal Allocation , 2019, IEEE Transactions on Information Forensics and Security.

[12]  Ginestra Bianconi,et al.  Weighted Multiplex Networks , 2013, PloS one.

[13]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[14]  Leandros Tassiulas,et al.  Backbone formation in military multi-layer ad hoc networks using complex network concepts , 2016, MILCOM 2016 - 2016 IEEE Military Communications Conference.

[15]  Binchao Huang,et al.  Identifying influential links to control spreading of epidemics , 2021 .

[16]  Michael I. Jordan,et al.  Is Q-learning Provably Efficient? , 2018, NeurIPS.

[17]  Evgeny Burnaev,et al.  Reinforcement Learning for Combinatorial Optimization: A Survey , 2020, ArXiv.

[18]  Yamir Moreno,et al.  A Multilayer perspective for the analysis of urban transportation systems , 2016, Scientific Reports.

[19]  Stefano Panzieri,et al.  Finding critical nodes in infrastructure networks , 2017, Int. J. Crit. Infrastructure Prot..

[20]  Ruchi Mittal,et al.  Anomaly Detection in Multiplex Networks , 2018 .

[21]  Francisco J. Rodríguez,et al.  Optimizing network attacks by artificial bee colony , 2017, Inf. Sci..

[22]  Hernán A. Makse,et al.  Collective Influence Algorithm to find influencers via optimal percolation in massively large social media , 2016, Scientific Reports.

[23]  Lenka Zdeborová,et al.  Fast and simple decycling and dismantling of networks , 2016, Scientific Reports.

[24]  Thomas C. Sharkey,et al.  Integrative Analytics for Detecting and Disrupting Transnational Interdependent Criminal Smuggling, Money, and Money-Laundering Networks , 2018, 2018 IEEE International Symposium on Technologies for Homeland Security (HST).

[25]  Eric B. Laber,et al.  Q-Learning: Theory and Applications , 2020, Annual Review of Statistics and Its Application.

[26]  Lin Wang,et al.  Evolutionary games on multilayer networks: a colloquium , 2015, The European Physical Journal B.

[27]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[28]  Anthony Chen,et al.  Performance of transportation network under perturbations: Reliability, vulnerability, and resilience , 2020 .

[29]  L. da F. Costa,et al.  Characterization of complex networks: A survey of measurements , 2005, cond-mat/0505185.

[30]  Matthias Dehmer,et al.  Interplay between SIR-based disease spreading and awareness diffusion on multiplex networks , 2018, J. Parallel Distributed Comput..

[31]  Makarand Hastak,et al.  Social network analysis: Characteristics of online social networks after a disaster , 2018, Int. J. Inf. Manag..

[32]  Yusef Esa,et al.  Applications of Complex Network Analysis in Electric Power Systems , 2018, Energies.

[33]  M. Perc,et al.  Network science of biological systems at different scales: A review. , 2017, Physics of life reviews.

[34]  Jose F. F. Mendes,et al.  Targeted Damage to Interdependent Networks , 2018, Physical Review E.

[35]  Shannon L. Risacher,et al.  Network approaches to systems biology analysis of complex disease: integrative methods for multi-omics data , 2017, Briefings Bioinform..

[36]  Beom Jun Kim,et al.  Attack vulnerability of complex networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[37]  Jun Wu,et al.  Efficient disintegration strategy in directed networks based on tabu search , 2018 .

[38]  Zhiwu Li,et al.  Containment of rumor spread in complex social networks , 2020, Inf. Sci..

[39]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[40]  Zhendong Su,et al.  Disrupting Terrorist Networks Based on Link Prediction: A Case Study of the 9–11 Hijackers Network , 2019, IEEE Access.

[41]  Sergio Gómez,et al.  Random Walks on Multiplex Networks , 2013, ArXiv.

[42]  Zhuoran Yang,et al.  A Theoretical Analysis of Deep Q-Learning , 2019, L4DC.