Batch Sampling : Low Overhead Scheduling for Sub-Second Parallel Jobs

Large-scale data analytics frameworks are shifting towards shorter task durations and larger degrees of parallelism to provide low latency. However, scheduling highly parallel jobs that complete in hundreds of milliseconds poses a major scaling challenge for cluster schedulers, which will need to place millions of tasks per second on appropriate nodes while offering millisecondlevel latency and high availability. This paper presents a decentralized load balancing approach called batch sampling, based on a generalization of the power of two random choices, that performs within a few percent of an optimal centralized scheduler. We evaluate our approach through both analytical results and an implementation called Sparrow. Sparrow schedules tasks with less than 8ms of overhead and features a design that is inherently fault-tolerant and scalable.