Accessing multiple mirror sites in parallel: using Tornado codes to speed up downloads

Mirror sites enable client requests to be serviced by any of a number of servers, reducing load at individual servers and dispersing network load. Typically, a client requests service from a single mirror site. We consider enabling a client to access a file from multiple mirror sites in parallel to speed up the download. To eliminate complex client-server negotiations that a straightforward implementation of this approach would require, we develop a feedback-free protocol based on erasure codes. We demonstrate that a protocol using fast Tornado codes can deliver dramatic speedups at the expense of transmitting a moderate number of additional packets into the network. This scalable solution extends naturally to allow multiple clients to access data from multiple mirror sites simultaneously. The approach applies naturally to wireless networks and satellite networks as well.

[1]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[2]  Nicholas F. Maxemchuk,et al.  DISPERSITY ROUTING IN STORE-AND-FORWARD NETWORKS. , 1975 .

[3]  Michael O. Rabin,et al.  Efficient dispersal of information for security, load balancing, and fault tolerance , 1989, JACM.

[4]  M. Rabin,et al.  An Adaptive Information Dispersal Algorithm for Time-critical Reliable Communication , 1994 .

[5]  Azer Bestavros An Adaptive Information Dispersal Algorithm for Time-Critical Reliable Communication , 1994 .

[6]  ZHANGLi-xia,et al.  A reliable multicast framework for light-weight sessions and application level framing , 1995 .

[7]  Madhu Sudan,et al.  A reliable dissemination protocol for interactive collaborative applications , 1995, MULTIMEDIA '95.

[8]  Stanley B. Zdonik,et al.  Dissemination-based data delivery using broadcast disks , 1995, IEEE Wirel. Commun..

[9]  V. Jacobson,et al.  A reliable multicast framework for light-weight sessions and application level framing , 1995, SIGCOMM '95.

[10]  Marek Karpinski,et al.  An XOR-based erasure-resilient coding scheme , 1995 .

[11]  Steven McCanne,et al.  Receiver-driven layered multicast , 1996, SIGCOMM '96.

[12]  Christian Huitema,et al.  The case for packet level FEC , 1996, Protocols for High-Speed Networks.

[13]  Azer Bestavros,et al.  AIDA-based real-time fault-tolerant broadcast disks , 1996, Proceedings Real-Time Technology and Applications.

[14]  Mark Crovella,et al.  Dynamic Server Selection using Bandwidth Probing in Wide-Area Networks , 1996 .

[15]  Don Towsley,et al.  Packet loss correlation in the MBone multicast network , 1996, Proceedings of GLOBECOM'96. 1996 IEEE Global Telecommunications Conference.

[16]  Srinivasan Seshan,et al.  SPAND: Shared Passive Network Performance Discovery , 1997, USENIX Symposium on Internet Technologies and Systems.

[17]  TowsleyDon,et al.  Parity-based loss recovery for reliable multicast transmission , 1997 .

[18]  Luigi Rizzo,et al.  A reliable multicast data distribution protocol based on software FEC techniques , 1997, The Fourth IEEE Workshop on High-Performance Communication Systems.

[19]  Jim Gemmell,et al.  Using Multicast FEC to Solve the Midnight Madness Problem , 1997 .

[20]  Steven McCanne,et al.  Low-Complexity Video Coding for Receiver-Driven Layered Multicast , 1997, IEEE J. Sel. Areas Commun..

[21]  Daniel A. Spielman,et al.  Practical loss-resilient codes , 1997, STOC '97.

[22]  C. Kenneth Miller,et al.  Reliable multicast protocols: a practical view , 1997, Proceedings of 22nd Annual Conference on Local Computer Networks.

[23]  Luigi Rizzo,et al.  Effective erasure codes for reliable computer communication protocols , 1997, CCRV.

[24]  Ernst W. Biersack,et al.  Asynchronous Multicast Push: AMP , 1997 .

[25]  Donald F. Towsley,et al.  Parity-based loss recovery for reliable multicast transmission , 1997, TNET.

[26]  Kevin C. Almeroth,et al.  Scalable delivery of Web pages using cyclic best-effort multicast , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[27]  Jon Crowcroft,et al.  TCP-like congestion control for layered multicast data transfer , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[28]  Michael Luby,et al.  A digital fountain approach to reliable distribution of bulk data , 1998, SIGCOMM '98.

[29]  Michael Mitzenmacher,et al.  Analysis of random processes via And-Or tree evaluation , 1998, SODA '98.

[30]  Michael Mitzenmacher,et al.  The Power of Two Choices in Randomized Load Balancing , 2001, IEEE Trans. Parallel Distributed Syst..

[31]  Dispersity Routing,et al.  Dispersity Routing , .