Hound: Causal Learning for Datacenter-scale Straggler Diagnosis

Stragglers are exceptionally slow tasks within a job that delay its completion. Stragglers, which are uncommon within a single job, are pervasive in datacenters with many jobs. A large body of research has focused on mitigating datacenter stragglers, but relatively little research has focused on systematically and rigorously identifying their root causes. We present Hound, a statistical machine learning framework that infers the causes of stragglers from traces of datacenter-scale jobs. Hound is designed to achieve several objectives: datacenterscale diagnosis, interpretable models, unbiased inference, and computational efficiency. We demonstrate Hound’s capabilities for a production trace from Google’s warehouse-scale datacenters and two Spark traces from Amazon EC2 clusters.