The Effect of Heavy-Tailed Job Size Distributions on Computer System Design

Heavy-tailed distributions, a.k.a. power-law distributions, have been observed in many natural phenomena ranging from physical phenomena to sociological phenomena. Recently heavy-tailed distributions have been discovered in computer systems. In particular the sizes (service demands) of computing jobs have been found to exhibit a heavy-tailed (power-law) distribution. Most previous analytic work in the area of computer system design has assumed that job sizes (service demands) are exponentially distributed. Many of the policies, algorithms, and general rules-of-thumb which are currently used in computer systems originated from analyses which assumed an exponentially-distributed workload. In this paper we argue that we need to reevaluate existing computer system algorithms in light of the discovery of heavy-tailed workloads. We argue that an algorithm which is optimal under an exponentially distributed workload may be very far from optimal when the workload is heavy-tailed. We demonstrate this point via three common problems in the design of computer systems: 1. Choosing a migration policy in a network of workstations. 2. Choosing a task assignment policy for a distributed server. 3. Scheduling HTTP requests within a Web server. For each problem above, we show that the answer is highly dependent on the job size distribution. We show how to do analysis under heavy-tailed job size distributions. We then show that our analysis leads us to policies whose performance improves greatly over commonly-used solutions, in some cases by orders of magnitude. This paper is a compilation of a sequence of papers which the author co-wrote: 9, 6, 7, 8, 1]. Far more detail is contained in the original papers.