Hierarchical Dynamics, Interarrival Times, and Performance

We report on a model of the distribution of job submission interarrival times in supercomputers. Interarrival times are modeled as a consequence of a complicated set of decisions between users, the queuing algorithm, and other policies. This cascading hierarchy of decision-making processes leads to a particular kind of heavy-tailed distribution. Specifically, hierarchically constrained systems suggest that fatter tails are due to more levels coming into play in the overall decision-making process. The key contribution of this paper is that heavier tials resulting from more complex decision-making processes, that is more hierarchical levels, will lead to overall worse performance, even when the average interarrival time is the same. Finally, we offer some suggestions for how to overcome these issues and the tradeoffs involved.

[1]  Scott H. Clearwater,et al.  ASCI queuing systems: overview and comparisons , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[2]  Walter Willinger,et al.  Scaling phenomena in the Internet: Critically examining criticality , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Philip L. Rosenfeld,et al.  Fractal Nature of Software-Cache Interaction , 1983, IBM J. Res. Dev..

[4]  Scott H. Clearwater,et al.  Quelling queue storms , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[5]  Mor Harchol-Balter The Effect of Heavy-Tailed Job Size Distributions on Computer System Design , 1999 .

[6]  Honbo Zhou,et al.  The EASY - LoadLeveler API Project , 1996, JSSPP.

[7]  Eric A. Brewer,et al.  Self-similarity in file systems , 1998, SIGMETRICS '98/PERFORMANCE '98.

[8]  Sally Floyd,et al.  Wide-area traffic: the failure of Poisson modeling , 1994 .

[9]  Judy Kay,et al.  A fair share scheduler , 1988, CACM.

[10]  Walter Willinger,et al.  A Bibliographical Guide to Self-Similar Traffic and Performance Modeling for Modern High-Speed Netwo , 1996 .

[11]  D. Sornette,et al.  Stretched exponential distributions in nature and economy: “fat tails” with characteristic scales , 1998, cond-mat/9801293.

[12]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[13]  D. Sornette,et al.  Extreme Deviations and Applications , 1997, cond-mat/9705132.

[14]  J Klafter,et al.  On the relationship among three theories of relaxation in disordered systems. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Walter Willinger,et al.  Long-range dependence in variable-bit-rate video traffic , 1995, IEEE Trans. Commun..

[16]  R. Palmer,et al.  Models of hierarchically constrained dynamics for glassy relaxation , 1984 .