Adaptive Push-Pull: Disseminating Dynamic Web Data

An important issue in the dissemination of time-varying Web data such as sports scores and stock prices is the maintenance of temporal coherency. In the case of servers adhering to the HTTP protocol, clients need to frequently pull the data based on the dynamics of the data and a user's coherency requirements. In contrast, servers that possess push capability maintain state information pertaining to clients and push only those changes that are of interest to a user. These two canonical techniques have complementary properties with respect to the level of temporal coherency maintained, communication overheads, state space overheads, and loss of coherency due to (server) failures. In this paper, we show how to combine push and pull-based techniques to achieve the best features of both approaches. Our combined technique tailors the dissemination of data from servers to clients based on 1) the capabilities and load at servers and proxies and 2) clients' coherency requirements. Our experimental results demonstrate that such adaptive data dissemination is essential to meet diverse temporal coherency requirements, to be resilient to failures, and for the efficient and scalable utilization of server and network resources.

[1]  Margo I. Seltzer,et al.  World Wide Web Cache Consistency , 1996, USENIX Annual Technical Conference.

[2]  Guruduth Banavar,et al.  An efficient multicast protocol for content-based publish-subscribe systems , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[3]  Amin Vahdat,et al.  Design and evaluation of a continuous consistency model for replicated services , 2000, OSDI.

[4]  Farnam Jahanian,et al.  Salamander: A Push-based Distribution Substrate for Internet Applications , 1997, USENIX Symposium on Internet Technologies and Systems.

[5]  Daniel M. Dias,et al.  A scalable and highly available web server , 1996, COMPCON '96. Technologies for the Information Superhighway Digest of Papers.

[6]  Krithi Ramamritham,et al.  Maintaining temporal coherency of virtual data warehouses , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[7]  Prashant J. Shenoy,et al.  Adaptive leases: a strong consistency mechanism for the World Wide Web , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[8]  Arun Iyengar,et al.  Improving Web Server Performance by Caching Dynamic Data , 1997, USENIX Symposium on Internet Technologies and Systems.

[9]  Calton Pu,et al.  Continual Queries for Internet Scale Event-Driven Information Delivery , 1999, IEEE Trans. Knowl. Data Eng..

[10]  Chengjie Liu,et al.  Maintaining strong cache consistency in the World-Wide Web , 1997, Proceedings of 17th International Conference on Distributed Computing Systems.

[11]  Margo I. Seltzer,et al.  The case for geographical push-caching , 1995, Proceedings 5th Workshop on Hot Topics in Operating Systems (HotOS-V).

[12]  Steven McCanne,et al.  An active service framework and its application to real-time multimedia transcoding , 1998, SIGCOMM '98.

[13]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[14]  Michael Dahlin,et al.  Hierarchical Cache Consistency in a WAN , 1999, USENIX Symposium on Internet Technologies and Systems.

[15]  Peter B. Danzig,et al.  A Hierarchical Internet Object Cache , 1996, USENIX ATC.

[16]  Azer Bestavros,et al.  Speculative data dissemination and service to reduce server load, network traffic and service time in distributed information systems , 1996, Proceedings of the Twelfth International Conference on Data Engineering.

[17]  Stanley B. Zdonik,et al.  Balancing push and pull for data broadcast , 1997, SIGMOD '97.

[18]  Sandy Irani,et al.  Cost-Aware WWW Proxy Caching Algorithms , 1997, USENIX Symposium on Internet Technologies and Systems.

[19]  Amin Vahdat,et al.  Design and evaluation of a conit-based continuous consistency model for replicated services , 2002, TOCS.

[20]  David R. Cheriton,et al.  Leases: an efficient fault-tolerant mechanism for distributed file cache consistency , 1989, SOSP '89.

[21]  Eric A. Brewer,et al.  Adapting to network and client variation using infrastructural proxies: lessons and perspectives , 1998, IEEE Wirel. Commun..

[22]  Prashant J. Shenoy,et al.  Efficiently maintaining stock portfolios up-to-date on the Web , 2002, Proceedings Twelfth International Workshop on Research Issues in Data Engineering: Engineering E-Commerce/E-Business Systems RIDE-2EC 2002.