Data distribution algorithms for load balanced fault-tolerant Web access

Describes the design and analysis of RobustWeb, a scalable and fault-tolerant World Wide Web server cluster that is based on HTTP redirection. The system consists of a set of N back-end document servers and one or more redirection servers which receive the HTTP requests and redirect them to the document servers. A load distribution algorithm is used for initial distribution of the documents on the servers. Given a specific degree of replication k, the distribution algorithm guarantees that at least k replicas of each document are present after document distribution is complete. The redirection servers redirect requests to one of the replicas with a pre-computed redirection probability. When a server fails, the redirection probabilities are recomputed using a novel algorithm based on network flow. Theis enables the load to be approximately balanced among the remaining servers, allowing for graceful degradation of the service in the event of failures. A preliminary prototype of RobustWeb has been implemented.

[1]  Abdelsalam Heddaya,et al.  WebWave: globally load balanced fully distributed caching of hot published documents , 1997, Proceedings of 17th International Conference on Distributed Computing Systems.

[2]  Daniel M. Dias,et al.  A scalable and highly available web server , 1996, COMPCON '96. Technologies for the Information Superhighway Digest of Papers.

[3]  David E. Culler,et al.  Using smart clients to build scalable services , 1997 .

[4]  Ari Luotonen,et al.  World-Wide Web Proxies , 1994, Comput. Networks ISDN Syst..

[5]  Margo I. Seltzer,et al.  The case for geographical push-caching , 1995, Proceedings 5th Workshop on Hot Topics in Operating Systems (HotOS-V).

[6]  Dimitrios N. Serpanos,et al.  MMPacking: a load and storage balancing algorithm for distributed multimedia servers , 1998 .

[7]  Azer Bestavros,et al.  Using speculation to reduce server load and service time on the WWW , 1995, CIKM '95.

[8]  Michael Garland,et al.  Implementing distributed server groups for the World Wide Web , 1995 .

[9]  Michael F. Schwartz,et al.  Locating nearby copies of replicated Internet servers , 1995, SIGCOMM '95.

[10]  Mark Crovella,et al.  Dynamic Server Selection In The Internet , 1995, Third IEEE Workshop on the Architecture and Implementation of High Performance Communication Subsystems.

[11]  Yi-Min Wang,et al.  ONE-IP: Techniques for Hosting a Service on a Cluster of Machines , 1997, Comput. Networks.

[12]  Azer Bestavros,et al.  Speculative data dissemination and service to reduce server load, network traffic and service time in distributed information systems , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[13]  Oscar H. Ibarra,et al.  SWEB: towards a scalable World Wide Web server on multicomputers , 1996, Proceedings of International Conference on Parallel Processing.

[14]  Daniel A. Reed,et al.  NCSA's World Wide Web Server: Design and Performance , 1995, Computer.