Topology discovery of sparse random graphs with few participants

We consider the task of topology discovery of sparse random graphs using end‐to‐end random measurements (e.g., delay) between a subset of nodes, referred to as the participants. The rest of the nodes are hidden, and do not provide any information for topology discovery. We consider topology discovery under two routing models: (a) the participants exchange messages along the shortest paths and obtain end‐to‐end measurements, and (b) additionally, the participants exchange messages along the second shortest path. For scenario (a), our proposed algorithm results in a sub‐linear edit‐distance guarantee using a sub‐linear number of uniformly selected participants. For scenario (b), we obtain a much stronger result, and show that we can achieve consistent reconstruction when a sub‐linear number of uniformly selected nodes participate. This implies that accurate discovery of sparse random graphs is tractable using an extremely small number of participants. We finally obtain a lower bound on the number of participants required by any algorithm to reconstruct the original random graph up to a given edit distance. We also demonstrate that while consistent discovery is tractable for sparse random graphs using a small number of participants, in general, there are graphs which cannot be discovered by any algorithm even with a significant number of participants, and with the availability of end‐to‐end information along all the paths between the participants. © 2012 Wiley Periodicals, Inc. Random Struct. Alg., 2013

[1]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[2]  Horst Bunke,et al.  Inexact graph matching for structural pattern recognition , 1983, Pattern Recognit. Lett..

[3]  A. Dress,et al.  Reconstructing the shape of a tree from observed dissimilarity data , 1986 .

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[7]  Azriel Rosenfeld,et al.  Landmarks in Graphs , 1996, Discret. Appl. Math..

[8]  Y. Vardi,et al.  Network Tomography: Estimating Source-Destination Traffic Intensities from Link Data , 1996 .

[9]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[10]  B. Cheswick,et al.  The Internet mapping project , 1998 .

[11]  Michael I. Jordan Graphical Models , 1998 .

[12]  Tandy J. Warnow,et al.  A Few Logs Suffice to Build (almost) All Trees: Part II , 1999, Theor. Comput. Sci..

[13]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[14]  Ramesh Govindan,et al.  Heuristics for Internet map discovery , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[15]  Tao Jiang,et al.  A Polynomial Time Approximation Scheme for Inferring Evolutionary Trees from Quartet Topologies and Its Application , 2001, SIAM J. Comput..

[16]  Alfred O. Hero,et al.  Unicast inference of network link delay distributions from edge measurements , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[17]  M. Jovanovi MODELING PEER-TO-PEER NETWORK TOPOLOGIES THROUGH “ SMALL-WORLD ” MODELS AND POWER LAWS , 2001 .

[18]  David Shallcross,et al.  Distance Realization Problems with Applications to Internet Tomography , 2001, J. Comput. Syst. Sci..

[19]  Fan Chung Graham,et al.  The Diameter of Sparse Random Graphs , 2001, Adv. Appl. Math..

[20]  S H Strogatz,et al.  Random graph models of social networks , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[21]  Adilson E Motter,et al.  Cascade-based attacks on complex networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Fangzhe Chang,et al.  Topology inference in the presence of anonymous routers , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[23]  Ratul Mahajan,et al.  Measuring ISP topologies with rocketfuel , 2002, TNET.

[24]  Lada A. Adamic,et al.  Information flow in social groups , 2003, cond-mat/0305305.

[25]  Robert Nowak,et al.  Network Tomography: Recent Developments , 2004 .

[26]  Noga Alon,et al.  The Probabilistic Method, Second Edition , 2004 .

[27]  Cristopher Moore,et al.  On the bias of traceroute sampling: Or, power-law degree distributions in regular graphs , 2005, JACM.

[28]  Yuval Shavitt,et al.  DIMES: let the internet measure itself , 2005, CCRV.

[29]  Srikanth Kandula,et al.  Shrink: a tool for failure diagnosis in IP networks , 2005, MineNet '05.

[30]  Cristopher Moore,et al.  On the bias of traceroute sampling: or, power-law degree distributions in regular graphs , 2005, STOC '05.

[31]  Elchanan Mossel,et al.  Optimal phylogenetic reconstruction , 2005, STOC '06.

[32]  Walter Willinger,et al.  The many facets of internet topology and traffic , 2006, Networks Heterog. Media.

[33]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[34]  Thomas Erlebach,et al.  Network Discovery and Verification , 2005, IEEE Journal on Selected Areas in Communications.

[35]  Vijaya Ramachandran,et al.  The diameter of sparse random graphs , 2007, Random Struct. Algorithms.

[36]  Cynthia Dwork,et al.  Wherefore art thou r3579x?: anonymized social networks, hidden patterns, and structural steganography , 2007, WWW '07.

[37]  Paul Barford,et al.  Learning network structure from passive measurements , 2007, IMC '07.

[38]  Nikhil Srivastava,et al.  On the longest path algorithm for reconstructing trees from distance matrices , 2007, Inf. Process. Lett..

[39]  Nikhil Srivastava,et al.  Learning and Verifying Graphs Using Queries with a Focus on Edge Counting , 2007, ALT.

[40]  Thomas Erlebach,et al.  Approximate Discovery of Random Graphs , 2007, SAGA.

[41]  Jeong Han Kim,et al.  Optimal query complexity bounds for finding graphs , 2008, Artif. Intell..

[42]  M. Newman,et al.  Hierarchical structure and the prediction of missing links in networks , 2008, Nature.

[43]  Jure Leskovec,et al.  Statistical properties of community structure in large social and information networks , 2008, WWW.

[44]  Elchanan Mossel,et al.  Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms , 2007, SIAM J. Comput..

[45]  Anima Anandkumar,et al.  Tracking in a spaghetti bowl: monitoring transactions using footprints , 2008, SIGMETRICS '08.

[46]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[47]  Oleg E. Melnik,et al.  Encyclopedia of Complexity and Systems Science , 2008 .

[48]  Kamil Saraç,et al.  Resolving Anonymous Routers in Internet Topology Measurement Studies , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[49]  Kousha Etessami,et al.  Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations , 2005, JACM.

[50]  Michalis Faloutsos,et al.  Internet Topology , 2009, Encyclopedia of Complexity and Systems Science.

[51]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[52]  Robert A. Meyers,et al.  Encyclopedia of Complexity and Systems Science , 2009 .

[53]  Jure Leskovec,et al.  Predicting positive and negative links in online social networks , 2010, WWW '10.

[54]  Hanna Mazzawi,et al.  Optimally reconstructing weighted graphs using queries , 2010, SODA '10.

[55]  Dimitrios Gunopulos,et al.  Finding effectors in social networks , 2010, KDD.

[56]  Christos Faloutsos,et al.  Kronecker Graphs: An Approach to Modeling Networks , 2008, J. Mach. Learn. Res..

[57]  Jian Ni,et al.  Efficient and dynamic routing topology inference from end-to-end measurements , 2010, TNET.

[58]  Asuman E. Ozdaglar,et al.  Spread of (Mis)Information in Social Networks , 2009, Games Econ. Behav..

[59]  Ram Rajagopal,et al.  Network delay inference from additive metrics , 2006, Random Struct. Algorithms.

[60]  Devavrat Shah,et al.  Detecting sources of computer viruses in networks: theory and experiment , 2010, SIGMETRICS '10.

[61]  Jure Leskovec,et al.  On the Convexity of Latent Social Network Inference , 2010, NIPS.

[62]  Topology discovery of sparse random graphs with few participants , 2011, SIGMETRICS '11.

[63]  Jon M. Kleinberg,et al.  Wherefore art thou R3579X? , 2011, Commun. ACM.

[64]  Bernhard Schölkopf,et al.  Uncovering the Temporal Dynamics of Diffusion Networks , 2011, ICML.

[65]  Walking on a graph with a magnifying glass: stratified sampling via weighted random walks , 2011, PERV.

[66]  Nello Cristianini,et al.  Refining causality: who copied from whom? , 2011, KDD.

[67]  Minas Gjoka,et al.  Walking on a graph with a magnifying glass: stratified sampling via weighted random walks , 2011, PERV.

[68]  Jure Leskovec,et al.  Inferring networks of diffusion and influence , 2010, KDD.

[69]  Don Towsley,et al.  Multicast Topology Inference from End-to-end Measurements , 2013 .

[70]  Bernhard Schölkopf,et al.  Uncovering the structure and temporal dynamics of information propagation , 2014, Network Science.

[71]  Nicholas C. Wormald,et al.  The mixing time of the giant component of a random graph , 2006, Random Struct. Algorithms.