Characterizing machines lifecycle in Google data centers

Abstract Due to the increasing need for computational power, the market has shifted towards big centralized data centers. Understanding the nature of the dynamics of these data centers from machine and job/task perspective is critical to design efficient data center management policies like optimal resource/power utilization, capacity planning and optimal (reactive and proactive) maintenance scheduling. Whereas jobs/tasks dynamics have received a lot of attention, the study of the dynamics of the underlying machines supporting the jobs/tasks execution has received much less attention, even when these dynamics would substantially affect the performance of the jobs/tasks execution. Given the limited data available from large computing installations, only a few previous studies have inspected data centers and only concerning failures and their root causes. In this paper, we study the 2011 Google data center traces from the machine dynamics perspective. First, we characterize the machine events and their underlying distributions in order to have a better understanding of the entire machine lifecycle. Second, we propose a data-driven model to enable the estimate of the expected number of available machines at any instant of time. The model is parameterized and validated using the empirical data collected by Google during a one month period.