Data popularity and shortest-job-first scheduling of network transfers

This paper presents a strategy for scheduling transfers of data items from a single source to a single destination. We are motivated by the scenarios in which an author with limited connectivity relies on a remote publishing server for sharing data items with readers. Our strategy is based on the shortest job first (SJF) algorithm with the computational cost expressed in terms of the transfer time and the number of readers waiting for an item. We consider cases with static and dynamic popularity of items. We prove that our strategy is optimal in the static case. For the dynamic case, we use simulation results to demonstrate that our strategy results in lower average waiting time when compared to an SJF algorithm that bases its decision only on the expected transfer time or only on the popularity of a data item

[1]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[2]  Éva Tardos,et al.  Scheduling data transfers in a network and the set scheduling problem , 2003, J. Algorithms.

[3]  S. S. Panwalkar,et al.  A Survey of Scheduling Rules , 1977, Oper. Res..

[4]  Kihong Park,et al.  On the relationship between file sizes, transport protocols, and self-similar network traffic , 1996, Proceedings of 1996 International Conference on Network Protocols (ICNP-96).

[5]  Amin Vahdat,et al.  Bullet: high bandwidth data dissemination using an overlay mesh , 2003, SOSP '03.

[6]  Minos N. Garofalakis,et al.  Scheduling issues in multimedia query optimization , 1995, CSUR.

[7]  G. Rand Sequencing and Scheduling: An Introduction to the Mathematics of the Job-Shop , 1982 .

[8]  William L. Maxwell,et al.  Theory of scheduling , 1967 .

[9]  Chelliah Sriskandarajah,et al.  A Survey of Machine Scheduling Problems with Blocking and No-Wait in Process , 1996, Oper. Res..

[10]  Edward G. Coffman,et al.  Computer and job-shop scheduling theory , 1976 .

[11]  A. J. Clewett,et al.  Introduction to sequencing and scheduling , 1974 .

[12]  Ludmila Cherkasova,et al.  FastReplica: Efficient Large File Distribution Within Content Delivery Networks , 2003, USENIX Symposium on Internet Technologies and Systems.

[13]  Bruce M. Maggs,et al.  Globally Distributed Content Delivery , 2002, IEEE Internet Comput..

[14]  D. McNickle,et al.  Comparing Different Approaches to the use of DiffServ in the Internet , 2005, TENCON 2005 - 2005 IEEE Region 10 Conference.

[15]  Edward G. Coffman,et al.  Scheduling file transfers in a distributed network , 1983, PODC '83.

[16]  D. Atkin OR scheduling algorithms. , 2000, Anesthesiology.