A Taxonomy and Survey of Content Delivery Networks

Content Delivery Networks (CDNs) have evolved to overcome the inherent limitations of the Internet in terms of user perceived Quality of Service (QoS) when accessing Web content. A CDN replicates content from the origin server to cache servers, scattered over the globe, in order to deliver content to end-users in a reliable and timely manner from nearby optimal surrogates. Content distribution on the Internet has received considerable research attention. It combines development of high-end computing technologies with highperformance networking infrastructure and distributed replica management techniques. Therefore, our aim is to categorize and analyze the existing CDNs, and to explore the uniqueness, weaknesses, opportunities, and future directions in this field. In this paper, we provide a comprehensive taxonomy with a broad coverage of CDNs in terms of organizational structure, content distribution mechanisms, request redirection techniques, and performance measurement methodologies. We study the existing CDNs in terms of their infrastructure, request-routing mechanisms, content replication techniques, load balancing, and cache management. We also provide an indepth analysis and state-of-the-art survey of CDNs. Finally, we apply the taxonomy to map various CDNs. The mapping of the taxonomy to the CDNs helps in “gap” analysis in the content networking domain. It also provides a means to identify the present and future development in this field and validates the applicability and accuracy of the taxonomy.

[1]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[2]  Jon Postel,et al.  User Datagram Protocol , 1980, RFC.

[3]  Data processing-open systems interconnection - basic reference model , 1981, CCRV.

[4]  Stefano Ceri,et al.  Distributed Databases: Principles and Systems , 1984 .

[5]  Patrick Valduriez,et al.  Principles of Distributed Database Systems , 1990 .

[6]  Brewster Kahle,et al.  An information system for corporate users: wide area information servers , 1991 .

[7]  Craig Partridge,et al.  Host Anycasting Service , 1993, RFC.

[8]  Gary Scott Malkin,et al.  Traceroute Using an IP Option , 1993, RFC.

[9]  Matt Ganis,et al.  SOCKS Protocol Version 5 , 1996, RFC.

[10]  Yair Bartal,et al.  Probabilistic approximation of metric spaces and its algorithmic applications , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[11]  David R. Karger,et al.  Consistent hashing and random trees: distributed caching protocols for relieving hot spots on the World Wide Web , 1997, STOC '97.

[12]  Roy T. Fielding,et al.  Hypertext Transfer Protocol - HTTP/1.1 , 1997, RFC.

[13]  Syam Gadde,et al.  Reduce, reuse, recycle: an approach to building large Internet caches , 1997, Proceedings. The Sixth Workshop on Hot Topics in Operating Systems (Cat. No.97TB100133).

[14]  Duane Wessels,et al.  Internet Cache Protocol (ICP), version 2 , 1997, RFC.

[15]  Steffen Rothkugel,et al.  Enhancing the Web's Infrastructure: From Caching to Replication , 1997, IEEE Internet Comput..

[16]  David E. Culler,et al.  Using smart clients to build scalable services , 1997 .

[17]  Amit Aggarwal,et al.  Performance of Dynamic Replication Schemes for an Internet Hosting Service , 1998 .

[18]  Duane Wessels,et al.  Cache Digests , 1998, Comput. Networks.

[19]  Keith Ross,et al.  Cache Array Routing Protocol v1.1 , 1998 .

[20]  Erich M. Nahum,et al.  Locality-aware request distribution in cluster-based network servers , 1998, ASPLOS VIII.

[21]  Ellen W. Zegura,et al.  A novel server selection technique for improving the response time of a replicated service , 1998, Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98.

[22]  Reagan Moore,et al.  Data-intensive computing and digital libraries , 1998, CACM.

[23]  Jia Wang,et al.  A survey of web caching schemes for the Internet , 1999, CCRV.

[24]  Bo Li,et al.  On the optimal placement of web proxies in the Internet , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[25]  David R. Karger,et al.  Web Caching with Consistent Hashing , 1999, Comput. Networks.

[26]  P. Lebrun THE LARGE HADRON COLLIDER, A MEGASCIENCE PROJECT , 2000 .

[27]  Jaideep Srivastava,et al.  Automatic personalization based on Web usage mining , 2000, CACM.

[28]  Lixia Zhang,et al.  On the placement of Internet instrumentation , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[29]  J. Frankel,et al.  The gnutella protocol specification v0.4 document revision 1.2 , 2000 .

[30]  Ian T. Foster,et al.  The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets , 2000, J. Netw. Comput. Appl..

[31]  Martin Arlitt,et al.  A workload characterization study of the 1998 World Cup Web site , 2000, IEEE Netw..

[32]  Duane Wessels,et al.  Hyper Text Caching Protocol (HTCP/0.0) , 2000, RFC.

[33]  P. Krishnan,et al.  The cache location problem , 2000, TNET.

[34]  Paul Francis,et al.  IDMaps: a global internet host distance estimation service , 2001, TNET.

[35]  Anees Shaikh,et al.  On the effectiveness of DNS-based server selection , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[36]  I. Lazar,et al.  Exploring content delivery networking , 2001 .

[37]  Syam Gadde,et al.  Web caching and content distribution: a view from the interior , 2001, Comput. Commun..

[38]  Lisa Amini Distribution Requirements for Content Internetworking , 2001 .

[39]  Dinesh C. Verma,et al.  Content Distribution Networks: An Engineering Approach , 2001 .

[40]  Yuval Shavitt,et al.  Constrained mirror placement on the Internet , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[41]  Balachander Krishnamurthy,et al.  On the use and performance of content distribution networks , 2001, IMW '01.

[42]  Lili Qiu,et al.  On the placement of Web server replicas , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[43]  Brian D. Davison A Web Caching Primer , 2001, IEEE Internet Comput..

[44]  Leandro Navarro-Moldes,et al.  Improving the service time of web clients using server redirection , 2001, PERV.

[45]  James F. Doyle,et al.  Peer-to-Peer: harnessing the power of disruptive technologies , 2001, UBIQ.

[46]  Kirk L. Johnson,et al.  The measured performance of content distribution networks , 2001, Comput. Commun..

[47]  Wei-Ying Ma,et al.  Content Services Network: The Architecture and Protocols , 2001 .

[48]  Ian Cooper,et al.  Internet Web Replication and Caching Taxonomy , 2001, RFC.

[49]  Larry L. Peterson,et al.  Proceedings of the 5th Symposium on Operating Systems Design and Implementation the Effectiveness of Request Redirection on Cdn Robustness , 2022 .

[50]  Jussi Kangasharju,et al.  Object replication strategies in content distribution networks , 2002, Comput. Commun..

[51]  Balachander Krishnamurthy,et al.  Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites , 2002, WWW.

[52]  Aravind Srinivasan,et al.  Clustering and server selection using passive monitoring , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[53]  Alexander S. Szalay,et al.  The world-wide telescope , 2001, CACM.

[54]  Krishna P. Gummadi,et al.  An analysis of Internet content delivery systems , 2002, OPSR.

[55]  Philip S. Yu,et al.  The state of the art in locally distributed Web-server systems , 2002, CSUR.

[56]  Scott Shenker,et al.  Complex Queries in Dht-based Peer-to-peer Networks , 2002 .

[57]  Jia Wang,et al.  Proceedings of the 2002 Usenix Annual Technical Conference a Precise and Efficient Evaluation of the Proximity between Web Clients and Their Local Dns Servers , 2022 .

[58]  Karl Aberer,et al.  An Overview of Peer-to-Peer Information Systems , 2002, WDAS.

[59]  Bruce M. Maggs,et al.  Globally Distributed Content Delivery , 2002, IEEE Internet Comput..

[60]  Randy H. Katz,et al.  Dynamic Replica Placement for Scalable Content Delivery , 2002, IPTPS.

[61]  Pavlin Radoslavov,et al.  Topology-informed Internet replica placement , 2002, Comput. Commun..

[62]  Fred Douglis,et al.  CDN brokering , 2002, Comput. Commun..

[63]  B. Huffaker,et al.  Distance Metrics in the Internet , 2002, Anais do 2002 International Telecommunications Symposium.

[64]  Mark Green Content Internetworking Architectural Overview , 2002 .

[65]  Brad Cain,et al.  A Model for Content Internetworking (CDI) , 2003, RFC.

[66]  Novella Bartolini,et al.  A Walk through Content Delivery Networks , 2003, MASCOTS Tutorials.

[67]  Mark S. Day,et al.  Content Internetworking (CDI) Scenarios , 2003, RFC.

[68]  Libero Nigro,et al.  Collaborative Learning On-Demand on the Internet Mbone , 2003 .

[69]  Brad Cain,et al.  Known Content Network (CN) Request-Routing Mechanisms , 2003, RFC.

[70]  Jeffrey Considine,et al.  Simple Load Balancing for Distributed Hash Tables , 2003, IPTPS.

[71]  Zhen Xiao,et al.  Moving Edge-Side Includes to the Real Edge - the Clients , 2003, USENIX Symposium on Internet Technologies and Systems.

[72]  David R. Karger,et al.  Chord: a scalable peer-to-peer lookup protocol for internet applications , 2003, TNET.

[73]  Randy H. Katz,et al.  Efficient and adaptive Web replication using content clustering , 2003, IEEE J. Sel. Areas Commun..

[74]  David R. Karger,et al.  Looking up data in P2P systems , 2003, CACM.

[75]  George Pallis,et al.  Content Delivery Networks: Status and Trends , 2003, IEEE Internet Comput..

[76]  Jian Ni,et al.  Hierarchical content routing in large-scale multimedia content delivery network , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[77]  Michal Szymaniak,et al.  Netairt: A DNS-based Redirection System for Apache , 2003 .

[78]  Andrew B. Whinston,et al.  Scaling Web Services with Capacity Provision Networks , 2003, Computer.

[79]  Jintae Lee,et al.  An end-user perspective on file-sharing systems , 2003, CACM.

[80]  Larry L. Peterson,et al.  Reliability and Security in the CoDeeN Content Distribution Network , 2004, USENIX Annual Technical Conference, General Track.

[81]  Carlos E. Palau,et al.  Modeling content delivery networks and their performance , 2004, Comput. Commun..

[82]  David Mazières,et al.  Democratizing Content Publication with Coral , 2004, NSDI.

[83]  Reinaldo Penno,et al.  An Architecture for Open Pluggable Edge Services (OPES) , 2004, RFC.

[84]  Hilarie Orman,et al.  Policy, Authorization, and Enforcement Requirements of the Open Pluggable Edge Services (OPES) , 2004, RFC.

[85]  Anees Shaikh,et al.  A Web content serving utility , 2004, IBM Syst. J..

[86]  Larry L. Peterson,et al.  The dark side of the Web , 2004, Comput. Commun. Rev..

[87]  Michiaki Tatsubori,et al.  Efficient Web services response caching by selecting optimal data representation , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[88]  Michal Szymaniak,et al.  Replication for web hosting systems , 2004, CSUR.

[89]  Ramayya Krishnan,et al.  Optimal pricing of content delivery network (CDN) services , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[90]  Diomidis Spinellis,et al.  A survey of peer-to-peer content distribution technologies , 2004, CSUR.

[91]  Mikel Izal,et al.  Dissecting BitTorrent: Five Months in a Torrent's Lifetime , 2004, PAM.

[92]  Elisa Turrini,et al.  An architecture for content distribution internetworking , 2004 .

[93]  Gang Peng,et al.  CDN: Content Distribution Network , 2004, ArXiv.

[94]  Atsushi Iwata,et al.  Coarse-grain replica management strategies for dynamic replication of Web contents , 2004, Comput. Networks.

[95]  G. Fortino,et al.  The COMODIN System : A CDN-based Platform for Cooperative Media On-Demand on the InterNet * , 2004 .

[96]  Chun-Hsin Wu,et al.  Content Networks: Taxonomy and New Approaches , 2005, The Internet as a Large-Scale Complex System.

[97]  Onur Altintas,et al.  Scalable request routing with next-neighbor load sharing in multi-server environments , 2005, 19th International Conference on Advanced Information Networking and Applications (AINA'05) Volume 1 (AINA papers).

[98]  Yannis Manolopoulos,et al.  A latency-based object placement approach in content distribution networks , 2005, Third Latin American Web Congress (LA-WEB'2005).

[99]  Savio S. H. Tse Approximate algorithms for document placement in distributed Web servers , 2005, IEEE Transactions on Parallel and Distributed Systems.

[100]  Jian Ni,et al.  Large-scale cooperative caching and application-level multicast in multimedia content delivery networks , 2005, IEEE Communications Magazine.

[101]  Markus Hofmann,et al.  Content Networking - Architecture, Protocols, and Practice , 2005, The Morgan Kaufmann series in networking.

[102]  Jussara M. Almeida,et al.  Dynamic content distribution for mobile enterprise networks , 2005, IEEE Journal on Selected Areas in Communications.

[103]  Kihong Park,et al.  The Internet as a Large-Scale Complex System , 2005, Santa Fe Institute Studies in the Sciences of Complexity.

[104]  Spiridon Bakiras,et al.  Combining replica placement and caching techniques in content distribution networks , 2005, Comput. Commun..

[105]  C. Linnhoff-Popien,et al.  A profit maximizing distribution strategy for context-aware services , 2005, Second IEEE International Workshop on Mobile Commerce and Services.

[106]  George Pallis,et al.  A similarity based approach for integrated Web caching and content replication in CDNs , 2006, 2006 10th International Database Engineering and Applications Symposium (IDEAS'06).

[107]  George Pallis,et al.  Insight and perspectives for content delivery networks , 2006, CACM.

[108]  Ajay D. Kshemkalyani,et al.  Objective-optimal algorithms for long-term Web prefetching , 2006, IEEE Transactions on Computers.

[109]  R. Buyya,et al.  A Case for Peering of Content Delivery Networks , 2006 .

[110]  KyoungSoo Park,et al.  Scale and Performance in the CoBlitz Large-File Distribution Service , 2006, NSDI.

[111]  David Mazières,et al.  OASIS: Anycast for Any Service , 2006, NSDI.

[112]  Guillaume Pierre,et al.  Globule: a collaborative content delivery network , 2006, IEEE Communications Magazine.

[113]  Rajkumar Buyya,et al.  Decentralized media streaming infrastructure (DeMSI): An adaptive and high-performance peer-to-peer content delivery network , 2006, J. Syst. Archit..

[114]  Rajkumar Buyya,et al.  A taxonomy of Data Grids for distributed data sharing, management, and processing , 2005, CSUR.

[115]  George Pallis,et al.  Integrating Caching Techniques on a Content Distribution Network , 2006, ADBIS.

[116]  Yannis Manolopoulos,et al.  Replication Based on Objects Load under a Content Distribution Network , 2006, 22nd International Conference on Data Engineering Workshops (ICDEW'06).

[117]  Rajkumar Buyya,et al.  Data Replication Strategies in Wide-Area Distributed Systems , 2007 .

[118]  Rajkumar Buyya,et al.  Economy-based Content Replication for Peering Content Delivery Networks , 2006, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[119]  Gustavo Alonso,et al.  Analysis of Caching and Replication Strategies for Web Applications , 2007, IEEE Internet Computing.

[120]  Giancarlo Fortino,et al.  CDN-Supported Collaborative Media Streaming Control , 2007, IEEE MultiMedia.

[121]  Ramayya Krishnan,et al.  Service Adoption and Pricing of Content Delivery Network (CDN) Services , 2008, Manag. Sci..