Main-memory triangle computations for very large (sparse (power-law)) graphs

Finding, counting and/or listing triangles (three vertices with three edges) in massive graphs are natural fundamental problems, which have recently received much attention because of their importance in complex network analysis. Here we provide a detailed survey of proposed main-memory solutions to these problems, in a unified way. We note that previous authors have paid surprisingly little attention to space complexity of main-memory solutions, despite its both fundamental and practical interest. We therefore detail space complexities of known algorithms and discuss their implications. We also present new algorithms which are time optimal for triangle listing and beats previous algorithms concerning space needs. They have the additional advantage of performing better on power-law graphs, which we also detail. We finally show with an experimental study that these two algorithms perform very well in practice, allowing us to handle cases which were previously out of reach.

[1]  Noga Alon,et al.  Homomorphisms in Graph Property Testing - A Survey , 2005, Electron. Colloquium Comput. Complex..

[2]  Stefan Bornholdt,et al.  Handbook of Graphs and Networks: From the Genome to the Internet , 2003 .

[3]  Matthieu Latapy,et al.  Complex Network Measurements: Estimating the Relevance of Observed Properties , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[4]  Sebastiano Vigna,et al.  The webgraph framework I: compression techniques , 2004, WWW '04.

[5]  R. Milo,et al.  Network motifs in integrated cellular networks of transcription-regulation and protein-protein interaction. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Béla Bollobás,et al.  Mathematical results on scale‐free random graphs , 2005 .

[7]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[8]  Raphael Yuster,et al.  Fast sparse matrix multiplication , 2004, TALG.

[9]  Frank Harary,et al.  Toward a General Calculus of Phonemic Distribution , 1957 .

[10]  Mohammad Ghodsi,et al.  New Streaming Algorithms for Counting Triangles in Graphs , 2005, COCOON.

[11]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[12]  Ulrik Brandes,et al.  Network Analysis: Methodological Foundations , 2010 .

[13]  Sebastiano Vigna,et al.  UbiCrawler: a scalable fully distributed Web crawler , 2004, Softw. Pract. Exp..

[14]  Ziv Bar-Yossef,et al.  Reductions in streaming algorithms, with an application to counting triangles in graphs , 2002, SODA '02.

[15]  Raphael Yuster,et al.  Detecting short directed cycles using rectangular matrix multiplication and dynamic programming , 2004, SODA '04.

[16]  Matthieu Latapy,et al.  Measuring Fundamental Properties of Real-World Complex Networks , 2006, ArXiv.

[17]  Alon Itai,et al.  Finding a minimum circuit in a graph , 1977, STOC '77.

[18]  Frank Harary,et al.  Matrix measures for transitivity and balance , 1979 .

[19]  S. Shen-Orr,et al.  Networks Network Motifs : Simple Building Blocks of Complex , 2002 .

[20]  Dorothea Wagner,et al.  Finding, Counting and Listing All Triangles in Large Graphs, an Experimental Study , 2005, WEA.

[21]  B. Monien.  How to Find Long Paths Efficiently , 1985 .

[22]  S. Havlin,et al.  Breakdown of the internet under intentional attack. , 2000, Physical review letters.

[23]  Vladimir Batagelj,et al.  A subquadratic triad census algorithm for large sparse networks with small maximum degree , 2001, Soc. Networks.

[24]  Noga Alon,et al.  Finding and Counting Given Length Cycles (Extended Abstract) , 1994, ESA.

[25]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[26]  Dorothea Wagner,et al.  Approximating Clustering Coefficient and Transitivity , 2005, J. Graph Algorithms Appl..

[27]  Vladimir Batagelj,et al.  Pajek - Program for Large Network Analysis , 1999 .

[28]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[29]  Prabhakar Raghavan,et al.  Computing on data streams , 1999, External Memory Algorithms.

[30]  S N Dorogovtsev,et al.  Comment on "breakdown of the Internet under intentional attack". , 2001, Physical review letters.

[31]  Norishige Chiba,et al.  Arboricity and Subgraph Listing Algorithms , 1985, SIAM J. Comput..

[32]  Noga Alon,et al.  Finding and counting given length cycles , 1997, Algorithmica.

[33]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[34]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[35]  Sebastiano Vigna,et al.  The Webgraph framework II: codes for the World-Wide Web , 2004, Data Compression Conference, 2004. Proceedings. DCC 2004.

[36]  S. Shen-Orr,et al.  Network motifs: simple building blocks of complex networks. , 2002, Science.

[37]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[38]  Aravind Srinivasan,et al.  Structural and algorithmic aspects of massive social networks , 2004, SODA '04.

[39]  Gianni Franceschini,et al.  Radix Sorting with No Extra Space , 2007, ESA.

[40]  Petter Holme,et al.  Structure and time evolution of an Internet dating community , 2002, Soc. Networks.