Generating realistic scaled complex networks

Research on generative models plays a central role in the emerging field of network science, studying how statistical patterns found in real networks could be generated by formal rules. Output from these generative models is then the basis for designing and evaluating computational methods on networks including verification and simulation studies. During the last two decades, a variety of models has been proposed with an ultimate goal of achieving comprehensive realism for the generated networks. In this study, we (a) introduce a new generator, termed ReCoN; (b) explore how ReCoN and some existing models can be fitted to an original network to produce a structurally similar replica, (c) use ReCoN to produce networks much larger than the original exemplar, and finally (d) discuss open problems and promising research directions. In a comparative experimental study, we find that ReCoN is often superior to many other state-of-the-art network generation methods. We argue that ReCoN is a scalable and effective tool for modeling a given network while preserving important properties at both micro- and macroscopic scales, and for scaling the exemplar data by orders of magnitude in size.

[1]  P-M Binder,et al.  Frustration in Complexity , 2008, Science.

[2]  Ilya Safro,et al.  Multiscale network generation , 2012, 2015 18th International Conference on Information Fusion (Fusion).

[3]  Guido Caldarelli,et al.  Large Scale Structure and Dynamics of Complex Networks: From Information Technology to Finance and Natural Science , 2007 .

[4]  Sven Leyffer,et al.  Fast response to infection spread and cyber attacks on large-scale networks , 2012, J. Complex Networks.

[5]  Christian Schulz,et al.  Tree-Based Coarsening and Partitioning of Complex Networks , 2014, SEA.

[6]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  Jérôme Kunegis,et al.  KONECT: the Koblenz network collection , 2013, WWW.

[8]  Peter Sanders,et al.  Partitioning (hierarchically clustered) complex networks via size-constrained graph clustering , 2016, Journal of Heuristics.

[9]  Martin Nöllenburg,et al.  Drawing Large Graphs by Multilevel Maxent-Stress Optimization , 2015, Graph Drawing.

[10]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[11]  Matthieu Latapy,et al.  Efficient and simple generation of random simple connected graphs with prescribed degree sequence , 2005, J. Complex Networks.

[12]  Christos Faloutsos,et al.  Scalable modeling of real graphs using Kronecker multiplication , 2007, ICML '07.

[13]  F. Radicchi,et al.  Benchmark graphs for testing community detection algorithms. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  Christos Faloutsos,et al.  ANF: a fast and scalable tool for data mining in massive graphs , 2002, KDD.

[15]  David B. Skillicorn,et al.  Proceedings of the Fourth SIAM International Conference on Data Mining, Lake Buena Vista, Florida, USA, April 22-24, 2004 , 2004, SDM.

[16]  Uri Alon,et al.  Coarse-graining and self-dissimilarity of complex networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  Garry Robins,et al.  An introduction to exponential random graph (p*) models for social networks , 2007, Soc. Networks.

[18]  M. Newman,et al.  Network theory and SARS: predicting outbreak diversity , 2004, Journal of Theoretical Biology.

[19]  David H. Wolpert,et al.  Using self-dissimilarity to quantify complexity , 2007, Complex..

[20]  Christos Gkantsidis,et al.  The Markov Chain Simulation Method for Generating Connected Power Law Random Graphs , 2003, ALENEX.

[21]  Rok Sosic,et al.  SNAP , 2016, ACM Trans. Intell. Syst. Technol..

[22]  D L Brown,et al.  Modeling, Simulation and Analysis of Complex Networked Systems: A Program Plan for DOE Office of Advanced Scientific Computing Research , 2009 .

[23]  B. Bollobás The evolution of random graphs , 1984 .

[24]  Ilya Safro,et al.  Relaxation-based coarsening and multiscale graph organization , 2010, Multiscale Model. Simul..

[25]  Sergio Cavalieri,et al.  Simulation in the supply chain context: a survey , 2004, Comput. Ind..

[26]  Aravind Srinivasan,et al.  Modelling disease outbreaks in realistic urban social networks , 2004, Nature.

[27]  Jure Leskovec,et al.  {SNAP Datasets}: {Stanford} Large Network Dataset Collection , 2014 .

[28]  Dorothea Wagner,et al.  Structure-preserving sparsification methods for social networks , 2016, Social Network Analysis and Mining.

[29]  Peter Sanders,et al.  Recent Advances in Graph Partitioning , 2013, Algorithm Engineering.

[30]  Tamara G. Kolda,et al.  Mathematical Challenges in Cybersecurity , 2009 .

[31]  M. Newman,et al.  On the uniform generation of random graphs with prescribed degree sequences , 2003, cond-mat/0312028.

[32]  A. Brandt Multiscale Scientific Computation: Review 2001 , 2002 .

[33]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[34]  T. Snijders The statistical evaluation of social network dynamics , 2001 .

[35]  John Doyle,et al.  Complexity and robustness , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Peter Sanders,et al.  Better Approximation of Betweenness Centrality , 2008, ALENEX.

[37]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[38]  T. Vicsek,et al.  Uncovering the overlapping community structure of complex networks in nature and society , 2005, Nature.

[39]  Behrouz Homayoun Far,et al.  Verification of lack of emergent behavior in extending a social network of agents , 2014, Social Network Analysis and Mining.

[40]  Mason A. Porter,et al.  Social Structure of Facebook Networks , 2011, ArXiv.

[41]  Christian Staudt,et al.  Engineering Parallel Algorithms for Community Detection in Massive Networks , 2013, IEEE Transactions on Parallel and Distributed Systems.

[42]  D. Ron,et al.  Multigrid Solvers and Multilevel Optimization Strategies , 2003 .

[43]  Jure Leskovec,et al.  Statistical properties of community structure in large social and information networks , 2008, WWW.

[44]  R. Rothenberg,et al.  Risk network structure in the early epidemic phase of HIV transmission in Colorado Springs , 2002, Sexually transmitted infections.

[45]  William L. Briggs,et al.  A multigrid tutorial, Second Edition , 2000 .

[46]  Christian Staudt,et al.  NetworKit: A tool suite for large-scale complex network analysis , 2014, Network Science.

[47]  Fan Chung Graham,et al.  A random graph model for massive graphs , 2000, STOC '00.

[48]  Christos Faloutsos,et al.  R-MAT: A Recursive Model for Graph Mining , 2004, SDM.

[49]  Tamara G. Kolda,et al.  A Scalable Generative Graph Model with Community Structure , 2013, SIAM J. Sci. Comput..

[50]  Shilpa Chakravartula,et al.  Complex Networks: Structure and Dynamics , 2014 .

[51]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[52]  Christos Faloutsos,et al.  Graph mining: Laws, generators, and algorithms , 2006, CSUR.

[53]  Andrea Lancichinetti,et al.  Benchmarks for testing community detection algorithms on directed and weighted graphs with overlapping communities. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[54]  Christos Faloutsos,et al.  Kronecker Graphs: An Approach to Modeling Networks , 2008, J. Mach. Learn. Res..

[55]  David A. Bader,et al.  Benchmarking for Graph Clustering and Partitioning , 2014, Encyclopedia of Social Network Analysis and Mining.

[56]  Shashank Khandelwal,et al.  Exploring biological network structure with clustered random networks , 2009, BMC Bioinformatics.

[57]  Ilya Safro,et al.  Generating Scaled Replicas of Real-World Complex Networks , 2016, COMPLEX NETWORKS.

[58]  M. Omizo,et al.  Modeling , 1983, Encyclopedic Dictionary of Archaeology.

[59]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[60]  Weihua An,et al.  Fitting ERGMs on big networks. , 2016, Social science research.

[61]  László Lovász,et al.  Multifractal network generator , 2010, Proceedings of the National Academy of Sciences.

[62]  S. Redner,et al.  Organization of growing random networks. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[63]  Ulrik Brandes,et al.  Efficient generation of large random networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[64]  P-M Binder Reflections on a Wall of Light , 2008, Science.

[65]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[66]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[67]  Katharina Anna Zweig,et al.  Different flavors of randomness: comparing random graph models with fixed degree sequences , 2015, Social Network Analysis and Mining.

[68]  Edoardo M. Airoldi,et al.  A Survey of Statistical Network Models , 2009, Found. Trends Mach. Learn..

[69]  Henning Meyerhenke,et al.  Generating Random Hyperbolic Graphs in Subquadratic Time , 2015, ISAAC.

[70]  Priya Mahadevan,et al.  Systematic topology analysis and generation using degree correlations , 2006, SIGCOMM.

[71]  Christian Staudt Algorithms and Software for the Analysis of Large Complex Networks , 2016 .

[72]  Ibrahim Matta,et al.  On the origin of power laws in Internet topologies , 2000, CCRV.

[73]  Gerth Stølting Brodal,et al.  2007 Proceedings of the Ninth Workshop on Algorithm Engineering and Experiments (ALENEX) , 2007 .

[74]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[75]  M. Keeling,et al.  Modeling Infectious Diseases in Humans and Animals , 2007 .

[76]  Tamara G. Kolda,et al.  Community structure and scale-free collections of Erdös-Rényi graphs , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[77]  Desh Ranjan,et al.  ParK: An efficient algorithm for k-core decomposition on multicore processors , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[78]  Enys Mones,et al.  Hierarchy Measure for Complex Networks , 2012, PloS one.