The determinant of performance in scale-up graph processing on a single system is the speed at which the graph can be fetched from storage: either from disk into memory or from memory into CPU-cache. Algorithms that follow edges perform random accesses to the storage medium for the graph and this can often be the determinant of performance, regardless of the algorithmic complexity or runtime efficiency of the actual algorithm in use. A storage-centric viewpoint would suggest that the solution to this problem lies in recognizing that graphs represent a unique workload and therefore should be treated as such by adopting novel ways to access graph structured data. We approach this problem from two different aspects and this paper details two different efforts in this direction. One approach is specific to graphs stored on SSDs and accelerates random access using a novel prefetcher called RASP. The second approach takes a fresh look at how graphs are accessed and suggests that trading off the low cost of random access for the approach of sequentially streaming a large set of (potentially unrelated) edges can be a winning proposition under certain circumstances: leading to a system for graphs stored on any medium (main-memory, SSD or magnetic disk) called X-stream. RASP and X-stream therefore take - diametrically opposite - storage centric viewpoints of the graph processing problem. After contrasting the approaches and demonstrating the benefit of each, this paper ends with a description of planned future development of an online algorithm that selects between the two approaches, possibly providing the best of both worlds.
[1]
Aart J. C. Bik,et al.
Pregel: a system for large-scale graph processing
,
2010,
SIGMOD Conference.
[2]
Joseph Gonzalez,et al.
PowerGraph: Distributed Graph-Parallel Computation on Natural Graphs
,
2012,
OSDI.
[3]
Abraham Bernstein,et al.
Signal/Collect: Graph Algorithms for the (Semantic) Web
,
2010,
SEMWEB.
[4]
Christos Faloutsos,et al.
Graph evolution: Densification and shrinking diameters
,
2006,
TKDD.
[5]
Hosung Park,et al.
What is Twitter, a social network or a news media?
,
2010,
WWW '10.
[6]
Valentin Dalibard,et al.
Mitigating I/O latency in SSD-based graph traversal
,
2012
.
[7]
Nancy M. Amato,et al.
Multithreaded Asynchronous Graph Traversal for In-Memory and Semi-External Memory
,
2010,
2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis.
[8]
David A. Bader,et al.
Scalable Graph Exploration on Multicore Processors
,
2010,
2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis.
[9]
Kunle Olukotun,et al.
Efficient Parallel Graph Exploration on Multi-Core CPU and GPU
,
2011,
2011 International Conference on Parallel Architectures and Compilation Techniques.
[10]
Guy E. Blelloch,et al.
GraphChi: Large-Scale Graph Computation on Just a PC
,
2012,
OSDI.
[11]
Kunle Olukotun,et al.
Green-Marl: a DSL for easy and efficient graph analysis
,
2012,
ASPLOS XVII.
[12]
Rajeev Motwani,et al.
The PageRank Citation Ranking : Bringing Order to the Web
,
1999,
WWW 1999.