Computing Exact Vertex Eccentricity on Massive-Scale Distributed Graphs

The eccentricity of a vertex is defined as the length of the longest shortest path to any other vertex. While eccentricity is an important measure of vertex centrality, directly computing exact eccentricity for all vertices on large-scale graphs is prohibitively costly. Takes and Kosters proposed an iterative algorithm that uses multiple runs of single-source shortest path (SSSP) to compute lower and upper bounds on eccentricity at every vertex. Their technique converges to exact eccentricity by performing SSSP from only a small percentage of vertices, when sources are efficiently selected. However, their source selection strategies do not always yield rapid convergence. We propose a pincer movement source selection algorithm that efficiently selects source vertices based on analysis of the lower and upper bounds produced by SSSP. We also leverage k-BFS, which runs breadth-first search (BFS) from multiple sources concurrently on HavoqGT, a high-performance vertex-centric message-passing graph processing framework, to achieve an additional significant performance improvement on distributed-memory systems. We demonstrate that our novel source vertex selection strategy has better performance on various real-world graph datasets compared with the previous strategy. In addition, we compute exact eccentricity for graphs with more than 1000X more edges (112B undirected edges) than graphs in the previous literature.

[1]  Nancy M. Amato,et al.  Scaling Techniques for Massive Scale-Free Graphs in Distributed (External) Memory , 2013, 2013 IEEE 27th International Symposium on Parallel and Distributed Processing.

[2]  Ümit V. Çatalyürek,et al.  Graph Manipulations for Fast Centrality Computation , 2017, ACM Trans. Knowl. Discov. Data.

[3]  Jure Leskovec,et al.  Defining and evaluating network communities based on ground-truth , 2012, Knowledge and Information Systems.

[4]  David A. Patterson,et al.  Distributed Memory Breadth-First Search Revisited: Enabling Bottom-Up Search , 2013, 2013 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum.

[5]  B. Yener,et al.  Cell-Graph Mining for Breast Tissue Modeling and Classification , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[6]  Julian Shun,et al.  An Evaluation of Parallel Eccentricity Estimation Algorithms on Undirected Real-World Graphs , 2015, KDD.

[7]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[8]  Tim Weninger,et al.  Thinking Like a Vertex , 2015, ACM Comput. Surv..

[9]  Nancy M. Amato,et al.  Faster Parallel Traversal of Scale Free Graphs at Extreme Scale with Vertex Delegates , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[10]  Ümit V. Çatalyürek,et al.  Regularizing graph centrality computations , 2015, J. Parallel Distributed Comput..

[11]  Jure Leskovec,et al.  {SNAP Datasets}: {Stanford} Large Network Dataset Collection , 2014 .

[12]  Jure Leskovec,et al.  Predicting positive and negative links in online social networks , 2010, WWW '10.

[13]  F. Harary,et al.  Eccentricity and centrality in networks , 1995 .

[14]  David A. Patterson,et al.  Direction-optimizing Breadth-First Search , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[15]  Andrei Z. Broder,et al.  Graph structure in the Web , 2000, Comput. Networks.

[16]  John D. Owens,et al.  Scalable Breadth-First Search on a GPU Cluster , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[17]  Jure Leskovec,et al.  Community Structure in Large Networks: Natural Cluster Sizes and the Absence of Large Well-Defined Clusters , 2008, Internet Math..

[18]  Jure Leskovec,et al.  Signed networks in social media , 2010, CHI.

[19]  Alfons Kemper,et al.  Parallel Array-Based Single- and Multi-Source Breadth First Searches on Large Dense Graphs , 2017, EDBT.

[20]  Walter A. Kosters,et al.  Determining the diameter of small world networks , 2011, CIKM '11.

[21]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[22]  Sebastiano Vigna,et al.  The Graph Structure in the Web - Analyzed on Different Aggregation Levels , 2015, J. Web Sci..

[23]  Danail Bonchev,et al.  The concept for the centre of a chemical structure and its applications , 1989 .

[24]  Walter A. Kosters,et al.  Computing the Eccentricity Distribution of Large Graphs , 2013, Algorithms.

[25]  Huy T. Vo,et al.  The More the Merrier: Efficient Multi-Source Graph Traversal , 2014, Proc. VLDB Endow..

[26]  Hosung Park,et al.  What is Twitter, a social network or a news media? , 2010, WWW '10.

[27]  Kamesh Madduri,et al.  Parallel breadth-first search on distributed memory systems , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[28]  Satoshi Matsuoka,et al.  Extreme scale breadth-first search on supercomputers , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[29]  Edmond Chow,et al.  A Scalable Distributed Parallel Breadth-First Search Algorithm on BlueGene/L , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[30]  A K Madan,et al.  Connective eccentricity index: a novel topological descriptor for predicting biological activity. , 2000, Journal of molecular graphics & modelling.