Topology discovery of sparse random graphs with few participants

We consider the task of topology discovery of sparse random graphs using end-to-end random measurements (e.g., delay) between a subset of nodes, referred to as the participants. The rest of the nodes are hidden, and do not provide any information for topology discovery. We consider topology discovery under two routing models: (a) the participants exchange messages along the shortest paths and obtain end-to-end measurements, and (b) additionally, the participants exchange messages along the second shortest path. For scenario (a), our proposed algorithm results in a sub-linear edit-distance guarantee using a sub-linear number of uniformly selected participants. For scenario (b), we obtain a much stronger result, and show that we can achieve consistent reconstruction when a sub-linear number of uniformly selected nodes participate. This implies that accurate discovery of sparse random graphs is tractable using an extremely small number of participants. We finally obtain a lower bound on the number of participants required by any algorithm to reconstruct the original random graph up to a given edit distance. We also demonstrate that while consistent discovery is tractable for sparse random graphs using a small number of participants, in general, there are graphs which cannot be discovered by any algorithm even with a significant number of participants, and with the availability of end-to-end information along all the paths between the participants.

[1]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[2]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[3]  Christos Faloutsos,et al.  Kronecker Graphs: An Approach to Modeling Networks , 2008, J. Mach. Learn. Res..

[4]  Jian Ni,et al.  Efficient and Dynamic Routing Topology Inference From End-to-End Measurements , 2010, IEEE/ACM Transactions on Networking.

[5]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[6]  Avinatan Hassidim,et al.  Topology discovery of sparse random graphs with few participants , 2011, SIGMETRICS '11.

[7]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[8]  Devavrat Shah,et al.  Detecting sources of computer viruses in networks: theory and experiment , 2010, SIGMETRICS '10.

[9]  Ram Rajagopal,et al.  Network delay inference from additive metrics , 2006, Random Struct. Algorithms.

[10]  Asuman Ozdaglar,et al.  Spread of (Mis)Information in Social Networks , 2009 .

[11]  S H Strogatz,et al.  Random graph models of social networks , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Jure Leskovec,et al.  Inferring networks of diffusion and influence , 2010, KDD.

[13]  Thomas Erlebach,et al.  Network Discovery and Verification , 2005, IEEE Journal on Selected Areas in Communications.

[14]  Robert Nowak,et al.  Network Tomography: Recent Developments , 2004 .

[15]  Jure Leskovec,et al.  On the Convexity of Latent Social Network Inference , 2010, NIPS.

[16]  Y. Vardi,et al.  Network Tomography: Estimating Source-Destination Traffic Intensities from Link Data , 1996 .

[17]  Azriel Rosenfeld,et al.  Landmarks in Graphs , 1996, Discret. Appl. Math..

[18]  Walter Willinger,et al.  The many facets of internet topology and traffic , 2006, Networks Heterog. Media.

[19]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[20]  Cristopher Moore,et al.  On the bias of traceroute sampling: or, power-law degree distributions in regular graphs , 2005, STOC '05.

[21]  Bernhard Schölkopf,et al.  Uncovering the Temporal Dynamics of Diffusion Networks , 2011, ICML.

[22]  David Shallcross,et al.  Distance Realization Problems with Applications to Internet Tomography , 2001, J. Comput. Syst. Sci..

[23]  Nikhil Srivastava,et al.  On the longest path algorithm for reconstructing trees from distance matrices , 2007, Inf. Process. Lett..

[24]  Yuval Shavitt,et al.  DIMES: let the internet measure itself , 2005, CCRV.

[25]  Fangzhe Chang,et al.  Topology inference in the presence of anonymous routers , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[26]  Minas Gjoka,et al.  Walking on a graph with a magnifying glass: stratified sampling via weighted random walks , 2011, PERV.

[27]  Alfred O. Hero,et al.  Unicast inference of network link delay distributions from edge measurements , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[28]  Vijaya Ramachandran,et al.  The diameter of sparse random graphs , 2007, Random Struct. Algorithms.

[29]  Elchanan Mossel,et al.  Reconstruction of Markov Random Fields from Samples: Some Observations and Algorithms , 2007, SIAM J. Comput..

[30]  Thomas Erlebach,et al.  Approximate Discovery of Random Graphs , 2007, SAGA.

[31]  Cristopher Moore,et al.  On the bias of traceroute sampling: Or, power-law degree distributions in regular graphs , 2005, JACM.

[32]  Michael I. Jordan Graphical Models , 1998 .

[33]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[34]  Horst Bunke,et al.  Inexact graph matching for structural pattern recognition , 1983, Pattern Recognit. Lett..

[35]  Dimitrios Gunopulos,et al.  Finding effectors in social networks , 2010, KDD.

[36]  Tandy J. Warnow,et al.  A Few Logs Suffice to Build (almost) All Trees: Part II , 1999, Theor. Comput. Sci..

[37]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[38]  M. Jovanovi MODELING PEER-TO-PEER NETWORK TOPOLOGIES THROUGH “ SMALL-WORLD ” MODELS AND POWER LAWS , 2001 .

[39]  Tao Jiang,et al.  A Polynomial Time Approximation Scheme for Inferring Evolutionary Trees from Quartet Topologies and Its Application , 2001, SIAM J. Comput..

[40]  Don Towsley,et al.  Multicast Topology Inference from End-to-end Measurements , 2013 .

[41]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[42]  Lada A. Adamic,et al.  Information flow in social groups , 2003, cond-mat/0305305.

[43]  Nello Cristianini,et al.  Refining causality: who copied from whom? , 2011, KDD.

[44]  Jure Leskovec,et al.  Statistical properties of community structure in large social and information networks , 2008, WWW.

[45]  Nikhil Srivastava,et al.  Learning and Verifying Graphs Using Queries with a Focus on Edge Counting , 2007, ALT.

[46]  Ramesh Govindan,et al.  Heuristics for Internet map discovery , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[47]  M. Newman,et al.  Hierarchical structure and the prediction of missing links in networks , 2008, Nature.

[48]  Adilson E Motter,et al.  Cascade-based attacks on complex networks. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[49]  Anima Anandkumar,et al.  Tracking in a spaghetti bowl: monitoring transactions using footprints , 2008, SIGMETRICS '08.

[50]  Srikanth Kandula,et al.  Shrink: a tool for failure diagnosis in IP networks , 2005, MineNet '05.

[51]  Kamil Saraç,et al.  Resolving Anonymous Routers in Internet Topology Measurement Studies , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[52]  Cynthia Dwork,et al.  Wherefore art thou r3579x?: anonymized social networks, hidden patterns, and structural steganography , 2007, WWW '07.

[53]  A. Dress,et al.  Reconstructing the shape of a tree from observed dissimilarity data , 1986 .

[54]  Michalis Faloutsos,et al.  Internet Topology , 2009, Encyclopedia of Complexity and Systems Science.

[55]  Elchanan Mossel,et al.  Optimal phylogenetic reconstruction , 2005, STOC '06.

[56]  Jeong Han Kim,et al.  Optimal query complexity bounds for finding graphs , 2008, Artif. Intell..

[57]  Jure Leskovec,et al.  Predicting positive and negative links in online social networks , 2010, WWW '10.

[58]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[59]  Paul Barford,et al.  Learning network structure from passive measurements , 2007, IMC '07.

[60]  Ratul Mahajan,et al.  Measuring ISP topologies with rocketfuel , 2002, TNET.

[61]  Hanna Mazzawi,et al.  Optimally reconstructing weighted graphs using queries , 2010, SODA '10.