H-SWEB: A Hierarchical Scheduling System for Distributed WWW Server Clusters

In this paper we present a model for dynamically scheduling HTTP requests across clusters of servers, optimizing the use of client resources as well as the scattered server nodes. We also present a system, H-SWEB, implementing our techniques and showing experimental improvements of over 250%, which have been achieved through utilizing a global approach to scheduling requests. This is the first system to provide a hierarchical scheduling mechanism for distributed HTTP server clusters incorporating dynamic client–server task distribution and distributed data access. H-SWEB uses sophisticated scheduling techniques in monitoring and adapting to workload variation at the client and server clusters for supporting typical digital library tasks, such as fast WWW image browsing. We provide a discussion of our system architecture and implementation, and briefly summarize the experimental and analytical results that have been achieved. Copyright 2000 John Wiley & Sons, Ltd.

[1]  Daniel Andresen,et al.  Towards a hierarchical scheduling system for distributed WWW server clusters , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[2]  Amara Lynn Graps,et al.  An introduction to wavelets , 1995 .

[3]  Eric A. Brewer,et al.  Reducing WWW Latency and Bandwidth Requirements by Real-Time Distillation , 1996, Comput. Networks.

[4]  Mark Crovella,et al.  Self-Similarity in World Wide Web Traffic: Evidence and Causes , 1996, SIGMETRICS.

[5]  Geoffrey Fox,et al.  Building a World-Wide Virtual Machine Based on Web and HPCC Technologies , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[6]  Joel H. Saltz,et al.  Sumatra: A Language for Resource-Aware Mobile Programs , 1996, Mobile Object Systems.

[7]  Oscar H. Ibarra,et al.  Toward a Scalable Distributed {WWW} Server on Workstation Clusters , 1997, J. Parallel Distributed Comput..

[8]  Daniel Andresen,et al.  Multiprocessor scheduling with client resources to improve the response time of WWW applications , 1997, ICS '97.

[9]  Tao Yang,et al.  Cooperative caching of dynamic content on a distributed Web server , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[10]  Oscar H. Ibarra,et al.  Experimental Studies on a Compact Storage Scheme for Wavelet-Based Multiresolution Subregion Retrieval , 1996, Data Compression Conference.

[11]  Oscar H. Ibarra,et al.  Adaptive Partitioning and Scheduling for Enhancing WWW Application Performance , 1998, J. Parallel Distributed Comput..

[12]  Tim Berners-Lee,et al.  Hypertext transfer protocol--http/i , 1993 .

[13]  James C. French,et al.  Legion: The Next Logical Step Toward a Nationwide Virtual Computer , 1994 .

[14]  Peter R. Cappello,et al.  Javelin: Internet‐based parallel computing using Java , 1997 .

[15]  Oscar H. Ibarra,et al.  SWEB: towards a scalable World Wide Web server on multicomputers , 1996, Proceedings of International Conference on Parallel Processing.

[16]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..

[17]  Daniel Andresen,et al.  Dynamic processor scheduling with client resources for fast multi-resolution WWW image browsing , 1997, Proceedings 11th International Parallel Processing Symposium.

[18]  Oscar H. Ibarra,et al.  The WWW Prototype of the Alexandria Digital Library , 1995 .

[19]  Martin F. Arlitt,et al.  Web server workload characterization: the search for invariants , 1996, SIGMETRICS '96.

[20]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[21]  Francine Berman,et al.  Application-Level Scheduling on Distributed Heterogeneous Networks , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.