A provably efficient algorithm for dynamic storage allocation

Abstract The design and analysis of algorithms for on-line dynamic storage allocation has been a fundamental problem area in computer science for many years. In this paper we study the stochastic behavior of dynamic allocation algorithms under the natural assumption that files enter and leave the system according to a Poisson process. In particular, we prove that for any dynamic allocation algorithm and any distribution of file sizes, the expected wasted space (or fragmentation) in the system at any time is Ω(√N √ log log N) , where N is the expected number of items (or used space) in the system. This result is known to be tight in the special case when all files have the same size. More importantly, we also construct a dynamic allocation algorithm which for any distribution of file sizes wastes only O(√N log 3 4 N) space with very high probability. This bound is also shown to be tight for a wide variety of file-size distributions, including for example the uniform and normal distributions. The results are significant because they show that the cumulative wasted space in the holes formed by the continual arrival and departure of items is a vanishingly small portion of the used space, at least on the average. This fact is in striking contrast with Knuth's well-known 50% rule which states that the number of these holes is linear in the used space. Moreover, the proof techniques establish a surprising connection between stochastic processes, such as dynamic allocation, and static problems such as bin-packing and planar matching. We suspect that the techniques will also prove useful in analyzing other stochastic processes which might otherwise prove intractable. Lastly, we present experimental data in support of the theoretical proofs, and as a basis for postulating several conjectures.

[1]  harald Cramer,et al.  Stationary And Related Stochastic Processes , 1967 .

[2]  Richard M. Karp,et al.  A probabilistic analysis of multidimensional bin packing problems , 1984, STOC '84.

[3]  P.W. Shor,et al.  The average-case analysis of some on-line algorithms for bin packing , 1984, Comb..

[4]  D. Iglehart.  Limiting diffusion approximations for the many server queue and the repairman problem , 1965 .

[5]  Frank Thomson Leighton,et al.  Tight bounds for minimax grid matching, with applications to the average case analysis of algorithms , 1986, STOC '86.

[6]  John E. Shore.  On the external storage fragmentation produced by first-fit and best-fit allocation strategies , 1975, CACM.

[7]  Edward G. Coffman,et al.  On the Asymptotic Optimality of First-Fit Storage Allocation , 1985, IEEE Transactions on Software Engineering.

[8]  John Michael Robson,et al.  Worst Case Fragmentation of First Fit and Best Fit Storage Allocation Strategies , 1977, Comput. J..

[9]  Gordon F. Newell,et al.  The M/M/⁰⁰ service system with ranked servers in heavy traffic , 1984 .

[10]  V. E. Beneš.  Models and Problems of Dynamic Memory Allocation , 1982 .

[11]  Edward G. Coffman,et al.  A Stochastic Model of Fragmentation in Dynamic Storage Allocation , 1985, SIAM J. Comput..

[12]  Jr. E. G. Coffman.  An Introduction to Combinatorial Models of Dynamic Storage Allocation , 1983 .

[13]  I. P. Page.  Optimal Fit Of Arbitrary Sized Segments , 1982, Comput. J..

[14]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[15]  H. Chernoff.  A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations , 1952 .

[16]  D. Iglehart.  Weak convergence of compound stochastic process, I , 1973 .

[17]  Thomas A. Standish.  Data Structure Techniques , 1980 .

[18]  Donald E. Knuth,et al.  fundamental algorithms , 1969 .

[19]  D. Aldous.  Some interesting processes arising as heavy traffic limits in an M/M/∞ storage process , 1986 .

[20]  S. Berman.  Stationary and Related Stochastic Processes , 1967 .

[21]  Edward G. Coffman,et al.  Algorithms for resolving conflicts in dynamic storage allocation , 1985, JACM.