Palgol: A High-Level DSL for Vertex-Centric Graph Processing with Remote Data Access

Pregel is a popular distributed computing model for dealing with large-scale graphs. However, it can be tricky to implement graph algorithms correctly and efficiently in Pregel’s vertex-centric model, especially when the algorithm has multiple computation stages, complicated data dependencies, or even communication over dynamic internal data structures. Some domain-specific languages (DSLs) have been proposed to provide more intuitive ways to implement graph algorithms, but due to the lack of support for remote access — reading or writing attributes of other vertices through references — they cannot handle the above mentioned dynamic communication, causing a class of Pregel algorithms with fast convergence impossible to implement.

[1]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[2]  Jennifer Widom,et al.  GPS: a graph processing system , 2013, SSDBM.

[3]  References , 1971 .

[4]  Joseph Gonzalez,et al.  PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs , 2012, OSDI.

[5]  Vladimir Vlassov,et al.  High-Level Programming Abstractions for Distributed Graph Processing , 2016, IEEE Transactions on Knowledge and Data Engineering.

[6]  Jian Zhai,et al.  A vertex centric parallel algorithm for linear temporal logic model checking in Pregel , 2014, J. Parallel Distributed Comput..

[7]  Jeffrey Xu Yu,et al.  Catch the Wind: Graph workload balancing on cloud , 2013, 2013 IEEE 29th International Conference on Data Engineering (ICDE).

[8]  Akimasa Morihata,et al.  Think like a vertex, behave like a function! a functional DSL for vertex-centric big graph processing , 2016, ICFP.

[9]  Wilfred Ng,et al.  Pregel Algorithms for Graph Connectivity Problems with Performance Guarantees , 2014, Proc. VLDB Endow..

[10]  Kunle Olukotun,et al.  Green-Marl: a DSL for easy and efficient graph analysis , 2012, ASPLOS XVII.

[11]  Kiminori Matsuzaki,et al.  s6raph: vertex-centric graph processing framework with functional interface , 2016, FHPC@ICFP.

[12]  Panos Kalnis,et al.  Mizan: a system for dynamic load balancing in large-scale graph processing , 2013, EuroSys '13.

[13]  Joseph Y. Halpern,et al.  Knowledge and common knowledge in a distributed environment , 1984, JACM.

[14]  Carlos Guestrin,et al.  Distributed GraphLab : A Framework for Machine Learning and Data Mining in the Cloud , 2012 .

[15]  Wilfred Ng,et al.  Blogel: A Block-Centric Framework for Distributed Computation on Real-World Graphs , 2014, Proc. VLDB Endow..

[16]  Anne Condon,et al.  Parallel implementation of Bouvka's minimum spanning tree algorithm , 1996, Proceedings of International Conference on Parallel Processing.

[17]  Robert E. Tarjan,et al.  A linear-time algorithm for a special case of disjoint set union , 1983, J. Comput. Syst. Sci..

[18]  HuZhenjiang,et al.  Think like a vertex, behave like a function! a functional DSL for vertex-centric big graph processing , 2016 .

[19]  Kunle Olukotun,et al.  Simplifying Scalable Graph Processing with a Domain-Specific Language , 2014, CGO '14.

[20]  David Hardcastle,et al.  Using Pregel-like Large Scale Graph Processing Frameworks for Social Network Analysis , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[21]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[22]  Lakshmish Ramaswamy,et al.  A distributed vertex-centric approach for pattern matching in massive graphs , 2013, 2013 IEEE International Conference on Big Data.

[23]  Michael Lesniak,et al.  Palovca: Describing and Executing Graph Algorithms in Haskell , 2012, PADL.

[24]  Jennifer Widom,et al.  Optimizing Graph Algorithms on Pregel-like Systems , 2014, Proc. VLDB Endow..

[25]  Shirish Tatikonda,et al.  From "Think Like a Vertex" to "Think Like a Graph" , 2013, Proc. VLDB Endow..