Reliable data distribution middleware for large-scale massive data replication

For massive data replication, Infocast, a communication middleware, is proposed. It distributes large data sets to thousands of receivers without any data loss. This paper examines the distributed processing feature of Infocast and also evaluates its performance. Reliability is achieved by an effective receiver-initiated recovery algorithm and server's complete supervision of all receiver states. Infocast provides reliable data distribution with high-performance to most receivers through the advantage of network-provided multicast while also supporting temporarily unavailable or performance impaired receivers by parallel state transitions. Implementation and performance test results including very large data distribution by VLDD scheme are examined through experiments. Scalability of receiver number and data size is discussed for various conditions.

[1]  Peter B. Danzig,et al.  Flow Control for Limited Buffer Multicast , 1994, IEEE Trans. Software Eng..

[2]  Willy Zwaenepoel,et al.  Distributed process groups in the V Kernel , 1985, TOCS.

[3]  Donald F. Towsley,et al.  A comparison of sender-initiated and receiver-initiated reliable multicast protocols , 1994, IEEE J. Sel. Areas Commun..

[4]  Alan O. Freier,et al.  Multicast Transport Protocol , 1992, RFC.

[5]  Stephen E. Deering,et al.  Multicast routing in datagram internetworks and extended LANs , 1990, TOCS.

[6]  Paul E. McKenney,et al.  Packet recovery in high-speed networks using coding and buffer management , 1990, Proceedings. IEEE INFOCOM '90: Ninth Annual Joint Conference of the IEEE Computer and Communications Societies@m_The Multiple Facets of Integration.

[7]  Louise E. Moser,et al.  Totem: a fault-tolerant multicast group communication system , 1996, CACM.

[8]  Donald F. Towsley,et al.  A Comparison of Sender-Initiated and Receiver-Initiated Reliable Multicast Protocols , 1997, IEEE J. Sel. Areas Commun..

[9]  Lixia Zhang,et al.  NETBLT: a high throughput transport protocol , 1987, Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication.

[10]  Peter B. Danzig,et al.  Massively Replicating Services in Wide-Area Internetworks , 1994 .

[11]  Jeremy R. Cooperstock,et al.  Why Use a Fishing Line When you Have a Net? An Adaptive Multicast Data Distribution Protocol , 1996, USENIX ATC.

[12]  Peter B. Danzig,et al.  Scalable Internet resource discovery: research problems and approaches , 1994, CACM.

[13]  Jo-Mei Chang,et al.  Reliable broadcast protocols , 1984, TOCS.