SPREAD: Scalable platform for reliable and efficient automated distribution

Abstract We introduce SPREAD — a new architecture for distributing and maintaining up-to-date Web content that simultaneously employs three different mechanisms: client validation, server invalidation, and replication. Proxies within SPREAD self-configure themselves to form scalable distribution hierarchies that connect the origin servers of content providers to clients. Each proxy autonomously decides on the best mechanism based on the object's popularity and modification rates. Requests and subscriptions propagate from edge proxies to the origin server through a chain of intermediate proxies. Invalidations and replications travel in the opposite direction. SPREAD's network of proxies automatically reconfigures when proxies go down or come up, or when new ones are added. The ability to spontaneously form hierarchies is based on a modified transparent proxying mechanism, called translucent proxying, that sanitizes transparent proxying. It allows proxies to be placed in an ad-hoc fashion anywhere in the network — not just at focal points within the network that are guaranteed to see all the packets of a TCP connection. In this paper we (1) describe the architecture of SPREAD, (2) discuss how proxies determine which mechanism to use based on local observations, and (3) use a trace-driven simulation to test SPREAD's behavior in a realistic setting.

[1]  Pablo Rodriguez,et al.  TPOT: translucent proxying of TCP , 2001, Comput. Commun..

[2]  James Gwertzman,et al.  Autonomous Replication in Wide-Area Internetworks , 1995 .

[3]  Ernst W. Biersack,et al.  Performance modelling of reliable multicast transmission , 1997, Proceedings of INFOCOM '97.

[4]  Martin F. Arlitt,et al.  Web server workload characterization: the search for invariants , 1996, SIGMETRICS '96.

[5]  Scott Shenker,et al.  A scalable Web cache consistency architecture , 1999, SIGCOMM '99.

[6]  Van Jacobson,et al.  Compressing TCP/IP Headers for Low-Speed Serial Links , 1990, RFC.

[7]  Pablo Rodriguez,et al.  Improving the WWW: Caching or Multicast? , 1998, Comput. Networks.

[8]  Stephen Pink,et al.  RFC 2507: IP header compression , 1999 .

[9]  P. Krishnan.  Transparent En-Route Caching in WANs? , 1999 .

[10]  Van Jacobson,et al.  Adaptive web caching: towards a new global caching architecture , 1998, Comput. Networks.

[11]  Hongsuda Tangmunarunkit,et al.  Scaling of multicast trees: comments on the Chuang-Sirbu scaling law , 1999, SIGCOMM '99.

[12]  Margo I. Seltzer,et al.  World Wide Web Cache Consistency , 1996, USENIX Annual Technical Conference.

[13]  Eric A. Brewer,et al.  System Design Issues for Internet Middleware Services: Deductions from a Large Client Trace , 1997, USENIX Symposium on Internet Technologies and Systems.

[14]  Chengjie Liu,et al.  Maintaining strong cache consistency in the World-Wide Web , 1997, Proceedings of 17th International Conference on Distributed Computing Systems.

[15]  Chengjie Liu,et al.  Maintaining Strong Cache Consistency in the World Wide Web , 1998, IEEE Trans. Computers.

[16]  Kurt Jeffery Worrell.  Invalidation in Large Scale Network Object Caches , 1994 .

[17]  Margo I. Seltzer,et al.  Autonomous replication across wide-area internetworks , 1995, SOSP.

[18]  Anja Feldmann,et al.  Rate of Change and other Metrics: a Live Study of the World Wide Web , 1997, USENIX Symposium on Internet Technologies and Systems.

[19]  Leonard Kleinrock,et al.  Queueing Systems: Volume I-Theory , 1975 .