Optimal Content Placement for a Large-Scale VoD System

IPTV service providers offering Video-on-Demand currently use servers at each metropolitan office to store all the videos in their library. With the rapid increase in library sizes, it will soon become infeasible to replicate the entire library at each office. We present an approach for intelligent content placement that scales to large library sizes (e.g., 100 Ks of videos). We formulate the problem as a mixed integer program (MIP) that takes into account constraints such as disk space, link bandwidth, and content popularity. To overcome the challenges of scale, we employ a Lagrangian relaxation-based decomposition technique combined with integer rounding. Our technique finds a near-optimal solution (e.g., within 1%-2%) with orders of magnitude speedup relative to solving even the linear programming (LP) relaxation via standard software. We also present simple strategies to address practical issues such as popularity estimation, content updates, short-term popularity fluctuation, and frequency of placement updates. Using traces from an operational system, we show that our approach significantly outperforms simpler placement strategies. For instance, our MIP-based solution can serve all requests using only half the link bandwidth used by least recently used (LRU) or least frequently used (LFU) cache replacement policies. We also investigate the tradeoff between disk space and network bandwidth.

[1]  Li Fan,et al.  Summary cache: a scalable wide-area web cache sharing protocol , 2000, TNET.

[2]  Sudipto Guha,et al.  Improved Combinatorial Algorithms for Facility Location Problems , 2005, SIAM J. Comput..

[3]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[4]  Carlos Maltzahn,et al.  Ceph: a scalable, high-performance distributed file system , 2006, OSDI '06.

[5]  Arun Venkataramani,et al.  Distributing content simplifies ISP traffic engineering , 2012, SIGMETRICS '13.

[6]  Sem C. Borst,et al.  Distributed Caching Algorithms for Content Distribution Networks , 2010, 2010 Proceedings IEEE INFOCOM.

[7]  Seungjoon Lee,et al.  Optimal Content Placement for a Large-Scale VoD System , 2016, TNET.

[8]  Sang Lyul Min,et al.  LRFU: A Spectrum of Policies that Subsumes the Least Recently Used and Least Frequently Used Policies , 2001, IEEE Trans. Computers.

[9]  Jochen Könemann,et al.  Faster and simpler algorithms for multicommodity flow and other fractional packing problems , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[10]  Songqing Chen,et al.  Does internet media traffic really follow Zipf-like distribution? , 2007, SIGMETRICS '07.

[11]  Pablo Rodriguez,et al.  I tube, you tube, everybody tubes: analyzing the world's largest user generated content video system , 2007, IMC '07.

[12]  Jaroslaw Byrka,et al.  An Optimal Bifactor Approximation Algorithm for the Metric Uncapacitated Facility Location Problem , 2006, SIAM J. Comput..

[13]  Bernardo A. Huberman,et al.  Predicting the Future with Social Media , 2010, 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[14]  Daniel Bienstock,et al.  Potential Function Methods for Approximately Solving Linear Programming Problems: Theory and Practice , 2002 .

[15]  Cheng-Zhong Xu,et al.  Optimal video replication and placement on a cluster of video-on-demand servers , 2002, Proceedings International Conference on Parallel Processing.

[16]  Seungjoon Lee,et al.  Anycast CDNS revisited , 2008, WWW.

[17]  Brad Cain,et al.  Known Content Network (CN) Request-Routing Mechanisms , 2003, RFC.

[18]  Mark Coates,et al.  Equipment allocation in video-on-demand network deployments , 2008, TOMCCAP.

[19]  Laurent Massoulié,et al.  Greening the internet with nano data centers , 2009, CoNEXT '09.

[20]  Lawrence W. Dowdy,et al.  Comparative Models of the File Assignment Problem , 1982, CSUR.

[21]  Mary K. Vernon,et al.  Network Bandwidth Requirements for Scalable On-Demand Streaming , 2007, IEEE/ACM Transactions on Networking.

[22]  Seungjoon Lee,et al.  Content Placement via the Exponential Potential Function Method , 2013, IPCO.

[23]  Chaitanya Swamy,et al.  Approximation Algorithms for Data Placement Problems , 2008, SIAM J. Comput..

[24]  Lili Qiu,et al.  On the placement of Web server replicas , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[25]  Cheng Huang,et al.  Can internet video-on-demand be profitable? , 2007, SIGCOMM '07.

[26]  Ratul Mahajan,et al.  Measuring ISP topologies with Rocketfuel , 2004, IEEE/ACM Transactions on Networking.

[27]  Alec Wolman,et al.  Volley: Automated Data Placement for Geo-Distributed Cloud Services , 2010, NSDI.