Local-Access Generators for Basic Random Graph Models

Consider a computation on a massive random graph: Does one need to generate the whole random graph up front, prior to performing the computation? Or, is it possible to provide an oracle to answer queries to the random graph "on-the-fly" in a much more efficient manner overall? That is, to provide a $local\ access\ generator$ which incrementally constructs the random graph locally, at the queried portions, in a manner consistent with the random graph model and all previous choices. Local access generators can be useful when studying the local behavior of specific random graph models. Our goal is to design local access generators whose required resource overhead for answering each query is significantly more efficient than generating the whole random graph. Our results focus on undirected graphs with independent edge probabilities, that is, each edge is chosen as an independent Bernoulli random variable. We provide a general implementation for generators in this model. Then, we use this construction to obtain the first efficient local implementations for the Erd\"{o}s-R\'{e}nyi $G(n,p)$ model, and the Stochastic Block model. As in previous local-access implementations for random graphs, we support $vertex$-$pair$, $next$-$neighbor$ queries, and $all$-$neighbors$ queries. In addition, we introduce a new $random$-$neighbor$ query. We also give the first local-access generation procedure for $all$-$neighbors$ queries in the (sparse and directed) Kleinberg's Small-World model. Note that, in the sparse case, an $all$-$neighbors$ query can be used to simulate the other types of queries efficiently. All of our generators require no pre-processing time, and answer each query using $polylog(n) $ time, random bits, and additional space.

[1]  Dana Ron,et al.  Property testing and its connection to learning and approximation , 1998, JACM.

[2]  Yishay Mansour,et al.  Converting Online Algorithms to Local Computation Algorithms , 2012, ICALP.

[3]  Dana Ron,et al.  Property Testing in Bounded Degree Graphs , 2002, STOC '97.

[4]  Emmanuel Abbe,et al.  Exact Recovery in the Stochastic Block Model , 2014, IEEE Transactions on Information Theory.

[5]  Maleq Khan,et al.  Parallel Algorithms for Generating Random Networks with Given Degree Sequences , 2014, International Journal of Parallel Programming.

[6]  M. Newman,et al.  On the uniform generation of random graphs with prescribed degree sequences , 2003, cond-mat/0312028.

[7]  Stéphane Bressan,et al.  Fast random graph generation , 2011, EDBT/ICDT '11.

[8]  R. Tibshirani,et al.  Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[9]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Aidong Zhang,et al.  Cluster analysis for gene expression data: a survey , 2004, IEEE Transactions on Knowledge and Data Engineering.

[11]  Sharon L. Milgram,et al.  The Small World Problem , 1967 .

[12]  Mark Newman,et al.  Models of the Small World , 2000 .

[13]  Charles U. Martel,et al.  Analyzing Kleinberg's (and other) small-world Models , 2004, PODC '04.

[14]  Kathryn B. Laskey,et al.  Stochastic blockmodels: First steps , 1983 .

[15]  D. Eisenberg,et al.  Detecting protein function and protein-protein interactions from genome sequences. , 1999, Science.

[16]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[17]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[18]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[19]  Michael Kearns,et al.  Local Algorithms for Finding Interesting Individuals in Large Networks , 2010, ICS.

[20]  L. Christophorou Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[21]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[22]  Ronitt Rubinfeld,et al.  Fast Local Computation Algorithms , 2011, ICS.

[23]  E. Abbe,et al.  Community detection and the stochastic block model , 2016 .

[24]  Oded Goldreich,et al.  On the Implementation of Huge Random Objects , 2003, SIAM J. Comput..

[25]  D. Watts,et al.  An Experimental Study of Search in Global Social Networks , 2003, Science.

[26]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[27]  Greg Linden,et al.  Amazon . com Recommendations Item-to-Item Collaborative Filtering , 2001 .

[28]  Silvio Micali,et al.  How to construct random functions , 1986, JACM.

[29]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[30]  Moni Naor,et al.  Algorithms and Models for the Web Graph , 2016, Lecture Notes in Computer Science.

[31]  Chris Arney,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World (Easley, D. and Kleinberg, J.; 2010) [Book Review] , 2013, IEEE Technology and Society Magazine.

[32]  Emmanuel Abbe,et al.  Community Detection in General Stochastic Block models: Fundamental Limits and Efficient Algorithms for Recovery , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[33]  Anup Rao,et al.  Stochastic Block Model and Community Detection in Sparse Graphs: A spectral algorithm with optimal rate of recovery , 2015, COLT.

[34]  O. Bagasra,et al.  Proceedings of the National Academy of Sciences , 1914, Science.

[35]  Elchanan Mossel,et al.  Reconstruction and estimation in the planted partition model , 2012, Probability Theory and Related Fields.

[36]  William W. Cohen,et al.  Community-Based Recommendations: a Solution to the Cold Start Problem , 2011 .

[37]  Boaz Patt-Shamir,et al.  On the Probe Complexity of Local Computation Algorithms , 2018, ICALP.

[38]  Danielle Smith Bassett,et al.  Small-World Brain Networks , 2006, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[39]  Ulrik Brandes,et al.  Efficient generation of large random networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[40]  E. David,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World , 2010 .

[41]  Norman W. Paton,et al.  Proceedings of the 16th International Conference on Extending Database Technology , 2013 .

[42]  Moti Medina,et al.  Sublinear Random Access Generators for Preferential Attachment Graphs , 2017, ICALP.

[43]  Joel C. Miller,et al.  Efficient Generation of Networks with Given Expected Degrees , 2011, WAW.

[44]  Aristotelis Tsirigos,et al.  Detecting community structures in Hi-C genomic data , 2015, 2016 Annual Conference on Information Science and Systems (CISS).

[45]  S H Strogatz,et al.  Random graph models of social networks , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[46]  Michael L. Creech,et al.  Integration of biological networks and gene expression data using Cytoscape , 2007, Nature Protocols.

[47]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[48]  Jingchun Chen,et al.  Detecting functional modules in the yeast protein-protein interaction network , 2006, Bioinform..

[49]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.