Dynamic Elasticity for Distributed Graph Analytics

Graph data analytics has received a great deal of attention in computing theory and systems research over the past decade. This paper proposes, implements and evaluates a distributed graph analytics system that scales resources up and down elastically to address the challenge of dynamic workload changes in analytics jobs. We show that by relying on hash partitioning, a simple and scalable method, and dynamically changing the placement of partitions from the already-partitioned graph, we can improve performance. Hence, the system eliminates the need to burden the user with resource acquisition and management decisions. We compare the system’s performance against a static partition placement to set expectations for real-world applications. Early evaluations show our system can achieve up to 2.4x speedup for certain applications.