Proceedings of the 5th Symposium on Operating Systems Design and Implementation an Analysis of Internet Content Delivery Systems

In the span of only a few years, the Internet has experienced an astronomical increase in the use of specialized content delivery systems, such as content delivery networks and peer-to-peer file sharing systems. Therefore, an understanding of content delivery on the Internet now requires a detailed understanding of how these systems are used in practice. This paper examines content delivery from the point of view of four content delivery systems: HTTP web traffic, the Akamai content delivery network, and Kazaa and Gnutella peer-to-peer file sharing traffic. We collected a trace of all incoming and outgoing network traffic at the University of Washington, a large university with over 60,000 students, faculty, and staff. From this trace, we isolated and characterized traffic belonging to each of these four delivery classes. Our results (1) quantify the rapidly increasing importance of new content delivery systems, particularly peer-to-peer networks, (2) characterize the behavior of these systems from the perspectives of clients, objects, and servers, and (3) derive implications for caching in these systems.

[1]  Balachander Krishnamurthy,et al.  On the use and performance of content distribution networks , 2001, IMW '01.

[2]  Anees Shaikh,et al.  On the effectiveness of DNS-based server selection , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[3]  Alec Wolman,et al.  Measurement and Analysis of a Streaming Media Workload , 2001, USITS.

[4]  Syam Gadde,et al.  Web caching and content distribution: a view from the interior , 2001, Comput. Commun..

[5]  Kirk L. Johnson,et al.  The measured performance of content distribution networks , 2001, Comput. Commun..

[6]  Jussi Kangasharju,et al.  Performance evaluation of redirection schemes in content distribution networks , 2001, Comput. Commun..

[7]  Li Fan,et al.  Summary cache: a scalable wide-area web cache sharing protocol , 2000, TNET.

[8]  Alec Wolman,et al.  Organization-Based Analysis of Web-Object Sharing and Caching , 1999, USENIX Symposium on Internet Technologies and Systems.

[9]  Jon M. Kleinberg,et al.  Mining the Web's Link Structure , 1999, Computer.

[10]  Jon M. Kleinberg,et al.  The Web as a Graph: Measurements, Models, and Methods , 1999, COCOON.

[11]  Michael Dahlin,et al.  Design considerations for distributed caching on the Internet , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[12]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[13]  Jin Zhang,et al.  Active Cache: caching dynamic contents on the Web , 1999, Distributed Syst. Eng..

[14]  Anja Feldmann,et al.  Web proxy caching: the devil is in the details , 1998, PERV.

[15]  Syam Gadde,et al.  Not all Hits are Created Equal: Cooperative Proxy Caching Over a Wide-Area Network , 1998, Comput. Networks.

[16]  Anja Feldmann,et al.  Rate of Change and other Metrics: a Live Study of the World Wide Web , 1997, USENIX Symposium on Internet Technologies and Systems.

[17]  Michael J. Feeley,et al.  The Measured Access Characteristics of World-Wide-Web Client Proxy Caches , 1997, USENIX Symposium on Internet Technologies and Systems.

[18]  Eric A. Brewer,et al.  System Design Issues for Internet Middleware Services: Deductions from a Large Client Trace , 1997, USENIX Symposium on Internet Technologies and Systems.

[19]  Virgílio A. F. Almeida,et al.  Measuring the behaviour of a world-wide web server , 1997, HPN.

[20]  Peter B. Danzig,et al.  A Hierarchical Internet Object Cache , 1996, USENIX ATC.

[21]  Steven McCanne,et al.  The BSD Packet Filter: A New Architecture for User-level Packet Capture , 1993, USENIX Winter.

[22]  Hector Garcia-Molina,et al.  Efficient search in peer to peer networks , 2004 .

[23]  Nathaniel Leibowitz,et al.  ARE FILE SWAPPING NETWORKS CACHEABLE? CHARACTERIZING P2P TRAFFIC , 2002 .

[24]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[25]  Geoffrey M. Voelker,et al.  The Medusa Proxy A Tool For Exploring User-Perceived Web Performance , 2001 .

[26]  Lili Qiu,et al.  The content and access dynamics of a busy Web site: findings and implications , 2000 .

[27]  G. Voelker,et al.  On the scale and performance of cooperative Web proxy caching , 2000, OPSR.

[28]  Ian Clarke,et al.  Freenet: A Distributed Anonymous Information Storage and Retrieval System , 2000, Workshop on Design Issues in Anonymity and Unobservability.