Faster Betweenness Centrality Based on Data Structure Experimentation

Abstract Betweenness centrality is a graph analytic that states the importance of a vertex based on the number of shortest paths that it is on. As such, betweenness centrality is a building block for graph analysis tools and is used by many applications, including finding bottlenecks in communication networks and community detection. Computing betweenness centrality is computation- ally demanding, O(V2 + V · E) (for the best known algorithm), which motivates the use of parallelism. Parallelism is especially needed for large graphs with millions of vertices and billions of edges. While the the memory requirements for computing be- tweenness are not as demanding, O(V + E) (for the best known sequential algorithm), these bound increase for different parallel algorithms. We show that is possible to reduce the memory requirements for computing betweenness centrality from O(V + E) to O(V) at the expense of doing additional traversals. We show that not only does this not hurt performance it actually improves performance for coarse grain parallelism. Further, we show that using the new approach allows parallel scaling that previously was not possible. One example is that the new approach is able to scale to 40 x86 cores for a graph with 32 M vertices and 2B edges, whereas the previous approach is only able to scale upto 6 cores because of memory requirements. We also do analysis of fine-grain parallel betweenness centrality on both the x86 and the Cray XMT.

[1]  David A. Bader,et al.  Parallel Algorithms for Evaluating Centrality Indices in Real-world Networks , 2006, 2006 International Conference on Parallel Processing (ICPP'06).

[2]  John R. Gilbert,et al.  The Combinatorial BLAS: design, implementation, and applications , 2011, Int. J. High Perform. Comput. Appl..

[3]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[4]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[5]  David A. Bader,et al.  Massive Social Network Analysis: Mining Twitter for Social Good , 2010, 2010 39th International Conference on Parallel Processing.

[6]  R. Guimerà,et al.  The worldwide air transportation network: Anomalous centrality, community structure, and cities' global roles , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[7]  B. Bollobás.  The evolution of random graphs , 1984 .

[8]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[9]  Andrei Z. Broder,et al.  Graph structure in the Web , 2000, Comput. Networks.

[10]  Albert-László Barabási,et al.  Internet: Diameter of the World-Wide Web , 1999, Nature.

[11]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[12]  David A. Bader,et al.  SNAP, Small-world Network Analysis and Partitioning: An open-source parallel graph framework for the exploration of large-scale networks , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[13]  Stephen Warshall,et al.  A Theorem on Boolean Matrices , 1962, JACM.

[14]  Olaf Sporns,et al.  Complex network measures of brain connectivity: Uses and interpretations , 2010, NeuroImage.

[15]  David A. Bader,et al.  National Laboratory Lawrence Berkeley National Laboratory Title A Faster Parallel Algorithm and Efficient Multithreaded Implementations for Evaluating Betweenness Centrality on Massive Datasets Permalink , 2009 .

[16]  Petr Konecny.  Introducing the Cray XMT , 2007 .

[17]  Sahin Albayrak,et al.  Application-level simulation for network security , 2008, SimuTools.

[18]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[19]  David A. Bader,et al.  Approximating Betweenness Centrality , 2007, WAW.

[20]  David F. Gleich,et al.  Algorithms and Models for the Web Graph , 2014, Lecture Notes in Computer Science.

[21]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Kate Nace Day.  Snap , 2003 .

[23]  Christos Faloutsos,et al.  R-MAT: A Recursive Model for Graph Mining , 2004, SDM.

[24]  Ninghui Sun,et al.  A Parallel Algorithm for Computing Betweenness Centrality , 2009, 2009 International Conference on Parallel Processing.

[25]  U. Brandes.  A faster algorithm for betweenness centrality , 2001 .

[26]  Guang R. Gao,et al.  Analysis and performance results of computing betweenness centrality on IBM Cyclops64 , 2009, The Journal of Supercomputing.

[27]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .