Improving parallel efficiency for asynchronous graph analytics using Gauss‐Seidel‐based matrix computation

Graph analytics is extensively used in big‐data applications such as social networks, web analysis, bio‐informatics, etc. Most graph processing frameworks adopt vertex‐centric model due to its ease of use and programming. However, when dealing with asynchronous graph analytics, frameworks based on vertex programming perform inefficiently. The reason is that first, vertex programming must guarantee the sequential consistency, which means frequent use of locks or atomic operations, and second, the algorithms are parallelized in vertex level and latent parallelism of the algorithms cannot be exploited. To improve parallel efficiency of asynchronous graph processing, the Gauss‐Seidel style algorithms in particular, this paper proposes a scheduling model using Gauss‐Seidel‐based matrix computation, which converts the vertex programming into two main matrix operations and then algorithms are parallelized by row and column vectors. Compared to vertex programming, our model parallelizes algorithms in a finer way to exploit more latent parallelism, while retains the ease‐of‐programming advantage of vertex programming. Instead of using locks to guarantee the sequential consistency, our model uses a hybrid synchronization policy to reduce serializability among threads and overheads of context switching. Furthermore, this model strengthens locality of the program. Experiment results show that our model outperforms vertex‐centric asynchronous frameworks in both performance and scalability. Moreover, it even surpasses the matrix‐based synchronous framework GraphMat with some non‐Gauss‐Seidel style algorithms.

[1]  Josep Torrellas,et al.  BulkSC: bulk enforcement of sequential consistency , 2007, ISCA '07.

[2]  Stephen J. Wright Coordinate descent algorithms , 2015, Mathematical Programming.

[3]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[4]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[5]  Taher H. Haveliwala Topic-sensitive PageRank , 2002, IEEE Trans. Knowl. Data Eng..

[6]  Guy E. Blelloch,et al.  GraphChi: Large-Scale Graph Computation on Just a PC , 2012, OSDI.

[7]  Kunle Olukotun,et al.  Green-Marl: a DSL for easy and efficient graph analysis , 2012, ASPLOS XVII.

[8]  Ali Tizghadam,et al.  A graph theoretical approach to traffic engineering and network control problem , 2009, 2009 21st International Teletraffic Congress.

[9]  Mohammad H. Al-Towaiq,et al.  Parallel Implementation of the Gauss-Seidel Algorithm on k-Ary n-Cube Machine , 2013 .

[10]  Scott McMillan,et al.  Design of the GraphBLAS API for C , 2017, 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[11]  Hai Jin,et al.  Frog: Asynchronous Graph Processing on GPU with Hybrid Coloring Model , 2018, IEEE Transactions on Knowledge and Data Engineering.

[12]  Pradeep Dubey,et al.  GraphMat: High performance graph analytics made productive , 2015, Proc. VLDB Endow..

[13]  Brian W. Barrett,et al.  Introducing the Graph 500 , 2010 .

[14]  Haibo Chen,et al.  SYNC or ASYNC: time to fuse for distributed graph-parallel computation , 2015, PPoPP.

[15]  Johannes Gehrke,et al.  Asynchronous Large-Scale Graph Processing Made Easy , 2013, CIDR.

[16]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[17]  Mikko H. Lipasti,et al.  Correctly implementing value prediction in microprocessors that support multithreading or multiprocessing , 2001, MICRO.

[18]  Ieee Staff 2013 IEEE High Performance Extreme Computing Conference (HPEC) , 2013 .

[19]  Jérémie Allard,et al.  Parallel Dense Gauss-Seidel Algorithm on Many-Core Processors , 2009, 2009 11th IEEE International Conference on High Performance Computing and Communications.

[20]  Joseph Gonzalez,et al.  PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs , 2012, OSDI.

[21]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[22]  Réka Albert,et al.  Near linear time algorithm to detect community structures in large-scale networks. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  Changjun Hu,et al.  A Cache-Efficient Parallel Gauss-Seidel Solver with Alternating Tiling , 2009, 2009 15th International Conference on Parallel and Distributed Systems.

[24]  John R. Gilbert,et al.  The Combinatorial BLAS: design, implementation, and applications , 2011, Int. J. High Perform. Comput. Appl..

[25]  Kadir Akbudak,et al.  Locality-Aware Parallel Sparse Matrix-Vector and Matrix-Transpose-Vector Multiplication on Many-Core Processors , 2016, IEEE Transactions on Parallel and Distributed Systems.

[26]  Leslie Lamport,et al.  How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs , 2016, IEEE Transactions on Computers.

[27]  Luigi Glielmo,et al.  A colored Gauss-Seidel approach for the distributed network flow problem , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[28]  Ming Wu,et al.  Managing Large Graphs on Multi-Cores with Graph Awareness , 2012, USENIX Annual Technical Conference.

[29]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[30]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[31]  Arthur Gretton,et al.  Parallel Gibbs Sampling: From Colored Fields to Thin Junction Trees , 2011, AISTATS.

[32]  Benno Schwikowski,et al.  Discovering regulatory and signalling circuits in molecular interaction networks , 2002, ISMB.

[33]  Geoffrey C. Fox,et al.  A parallel Gauss-Seidel algorithm for sparse power system matrices , 1994, Proceedings of Supercomputing '94.

[34]  Joy Arulraj,et al.  Apache Giraph , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[35]  Pradeep Dubey,et al.  Navigating the maze of graph analytics frameworks using massive graph datasets , 2014, SIGMOD Conference.

[36]  Michael Stonebraker,et al.  Standards for graph algorithm primitives , 2014, 2013 IEEE High Performance Extreme Computing Conference (HPEC).

[37]  Keshav Pingali,et al.  A lightweight infrastructure for graph analytics , 2013, SOSP.

[38]  Guy E. Blelloch,et al.  Ligra: a lightweight graph processing framework for shared memory , 2013, PPoPP '13.

[39]  Joseph M. Hellerstein,et al.  GraphLab: A New Framework For Parallel Machine Learning , 2010, UAI.