The STAPL Parallel Graph Library

This paper describes the stapl Parallel Graph Library, a high-level framework that abstracts the user from data-distribution and parallelism details and allows them to concentrate on parallel graph algorithm development. It includes a customizable distributed graph container and a collection of commonly used parallel graph algorithms. The library introduces pGraph pViews that separate algorithm design from the container implementation. It supports three graph processing algorithmic paradigms, level-synchronous, asynchronous and coarse-grained, and provides common graph algorithms based on them. Experimental results demonstrate improved scalability in performance and data size over existing graph libraries on more than 16,000 cores and on internet-scale graphs containing over 16 billion vertices and 250 billion edges.

[1]  Michael J. Quinn,et al.  Parallel graph algorithms , 1984, CSUR.

[2]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[3]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[4]  Afonso Ferreira,et al.  Efficient Parallel Graph Algorithms for Coarse-Grained Multicomputers and BSP , 2002, Algorithmica.

[5]  Edward W. Larsen,et al.  Fast iterative methods for discrete-ordinates particle transport calculations , 2002 .

[6]  Lawrence Rauchwerger,et al.  ARMI: an adaptive, platform independent communication library , 2003, PPoPP '03.

[7]  Nancy M. Amato,et al.  A framework for adaptive algorithm selection in STAPL , 2005, PPoPP.

[8]  Lawrence Rauchwerger,et al.  Finding strongly connected components in distributed graphs , 2005, J. Parallel Distributed Comput..

[9]  Nancy M. Amato,et al.  Parallel protein folding with STAPL , 2005, Concurr. Comput. Pract. Exp..

[10]  Douglas P. Gregor,et al.  The Parallel BGL : A Generic Library for Distributed Graph Computations , 2005 .

[11]  Jonathan W. Berry,et al.  Software and Algorithms for Graph Queries on Multithreaded Architectures , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[12]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[13]  Nancy M. Amato,et al.  Multithreaded Asynchronous Graph Traversal for In-Memory and Semi-External Memory , 2010, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis.

[14]  Nancy M. Amato,et al.  The STAPL pView , 2010, LCPC.

[15]  Nancy M. Amato,et al.  STAPL: standard template adaptive parallel library , 2010, SYSTOR '10.

[16]  Nancy M. Amato,et al.  The STAPL parallel container framework , 2011, PPoPP '11.

[17]  Wolfram Koepf,et al.  Lecture Notes in Computer Science (LNCS) , 2011 .

[18]  Sam Ade Jacobs,et al.  A scalable method for parallelizing sampling-based motion planning algorithms , 2012, 2012 IEEE International Conference on Robotics and Automation.

[19]  Kunle Olukotun,et al.  Green-Marl: a DSL for easy and efficient graph analysis , 2012, ASPLOS XVII.

[20]  K. Karlapalem,et al.  ET , 2013, Proceedings of the 22nd International Conference on World Wide Web - WWW '13 Companion.