Towards a hierarchical scheduling system for distributed WWW server clusters

We present a model for dynamically scheduling HTTP requests across clusters of servers, optimizing the use of client resources as well as the scattered server nodes. We also present a system, H-SWEB, implementing our techniques and showing experimental improvements of over 250%, which have been achieved through utilizing a global approach to scheduling requests. This is the first system to provide a hierarchical scheduling mechanism for distributed HTTP server clusters incorporating dynamic client-server task distribution and distributed data access. H-SWEB uses sophisticated scheduling techniques in monitoring and adapting to workload variation at the client and server clusters for supporting typical digital library tasks, such as fast WWW image browsing. We provide a discussion of our system architecture and implementation, and briefly summarize the experimental results that have been achieved.

[1]  Oscar H. Ibarra,et al.  The WWW Prototype of the Alexandria Digital Library , 1995 .

[2]  WolskiRich.  Dynamically forecasting network performance using the Network Weather Service , 1998 .

[3]  Joel H. Saltz,et al.  Sumatra: A Language for Resource-Aware Mobile Programs , 1996, Mobile Object Systems.

[4]  Daniel Andresen,et al.  Multiprocessor scheduling with client resources to improve the response time of WWW applications , 1997, ICS '97.

[5]  Charles K. Chui,et al.  An Introduction to Wavelets , 1992 .

[6]  James A. Gosling,et al.  The java language environment: a white paper , 1995 .

[7]  Edward A. Fox,et al.  Digital libraries , 1995, CACM.

[8]  Francine Berman,et al.  Application-Level Scheduling on Distributed Heterogeneous Networks , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[9]  Oscar H. Ibarra,et al.  Toward a Scalable Distributed {WWW} Server on Workstation Clusters , 1997, J. Parallel Distributed Comput..

[10]  Tim Berners-Lee,et al.  Hypertext transfer protocol--http/i , 1993 .

[11]  Eric A. Brewer,et al.  Reducing WWW Latency and Bandwidth Requirements by Real-Time Distillation , 1996, Comput. Networks.

[12]  James C. French,et al.  Legion: The Next Logical Step Toward a Nationwide Virtual Computer , 1994 .

[13]  Tao Yang,et al.  Cooperative caching of dynamic content on a distributed Web server , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[14]  Oscar H. Ibarra,et al.  Experimental Studies on a Compact Storage Scheme for Wavelet-Based Multiresolution Subregion Retrieval , 1996, Data Compression Conference.

[15]  Roy T. Fielding,et al.  Hypertext Transfer Protocol - HTTP/1.1 , 1997, RFC.

[16]  X. Evers.  Condor Flocking: Load Sharing between Pools of Workstations , 1993 .

[17]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[18]  Peter R. Cappello,et al.  Javelin: Internet-based Parallel Computing using Java , 1997, Concurr. Pract. Exp..

[19]  Daniel Andresen,et al.  Dynamic processor scheduling with client resources for fast multi-resolution WWW image browsing , 1997, Proceedings 11th International Parallel Processing Symposium.

[20]  Geoffrey Fox,et al.  Building a World-Wide Virtual Machine Based on Web and HPCC Technologies , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[21]  Peter R. Cappello,et al.  Javelin: Internet‐based parallel computing using Java , 1997 .

[22]  Oscar H. Ibarra,et al.  SWEB: towards a scalable World Wide Web server on multicomputers , 1996, Proceedings of International Conference on Parallel Processing.

[23]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..