High Performance On-demand Video Transcoding Using Cloud Services

Video streams, either in form of on-demand streaming or live streaming, usually have to be converted (i.e., transcoded) based on the characteristics (e.g., spatial resolution) of clients' devices. Transcoding is a computationally expensive operation, therefore, streaming service providers currently store numerous transcoded versions of the same video to serve different types of client devices. However, recent studies show that accessing video streams have a long tail distribution. That is, there are few popular videos that are frequently accessed while the majority of them are accessed infrequently. The idea we propose in this research is to transcode the infrequently accessed videos in a on-demand (i.e., lazy) manner. Due to the cost of maintaining infrastructure, streaming service providers (e.g., Netflix) are commonly using cloud services. However, the challenge in utilizing cloud services for video transcoding is how to deploy cloud resources in a cost-efficient manner without any major impact on the quality of video streams. To address the challenge, in this research, we present an architecture for on-demand transcoding of video streams. The architecture provides a platform for streaming service providers to utilize cloud resources in a cost-efficient manner and with respect to the Quality of Service (QoS) requirements of video streams. In particular, the architecture includes a QoS-aware scheduling component to efficiently map video streams to cloud resources, and a cost-efficient dynamic (i.e., elastic) resource provisioning policy that adapts the resource acquisition with respect to the video streaming QoS requirements.

[1]  Howard Jay Siegel,et al.  Representing Task and Machine Heterogeneities for Heterogeneous Computing Systems , 2000 .

[2]  Xinggong Zhang,et al.  Parallelizing video transcoding using Map-Reduce-based cloud computing , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[3]  Minghua Chen,et al.  CALMS: Cloud-assisted live media streaming for globalized demands with time/region diversities , 2012, 2012 Proceedings IEEE INFOCOM.

[4]  Ida-Maria Tunturipuro.  Building a Low-Cost Streaming System : Streaming and Camera Operating System for Live Internet Productions , 2015 .

[5]  Anthony A. Maciejewski,et al.  Maximizing stochastic robustness of static resource allocations in a periodic sensor driven cluster , 2014, Future Gener. Comput. Syst..

[6]  Daniel Lewis,et al.  What is web 2.0? , 2006, CROS.

[7]  Alexandru Iosup,et al.  The performance of bags-of-tasks in large-scale distributed systems , 2008, HPDC '08.

[8]  Alexandru Iosup,et al.  Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing , 2011, IEEE Transactions on Parallel and Distributed Systems.

[9]  A. Rowstron,et al.  Towards predictable datacenter networks , 2011, SIGCOMM.

[10]  Rajkumar Buyya,et al.  Adapting Market-Oriented Scheduling Policies for Cloud Computing , 2010, ICA3PP.

[11]  Sébastien Lafond,et al.  Prediction-Based Dynamic Resource Allocation for Video Transcoding in Cloud Computing , 2013, 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing.

[12]  Chen-Hsiu Huang.  Video Transcoding Architectures and Techniques : An Overview , 2003 .

[13]  Pangfeng Liu,et al.  SQLMR : A Scalable Database Management System for Cloud Computing , 2011, 2011 International Conference on Parallel Processing.

[14]  Anthony A. Maciejewski,et al.  Heuristics for Robust Resource Allocation of Satellite Weather Data Processing on a Heterogeneous Parallel System , 2011, IEEE Transactions on Parallel and Distributed Systems.

[15]  Anees Shaikh,et al.  Effective peering for multi-provider content delivery services , 2004, IEEE INFOCOM 2004.

[16]  Rajkumar Buyya,et al.  Statistical Modeling of Spot Instance Prices in Public Cloud Environments , 2011, 2011 Fourth IEEE International Conference on Utility and Cloud Computing.

[17]  Magdy A. Bayoumi,et al.  High-speed Motion Estimation Architecture for Real-time Video Transmission , 2012, Comput. J..

[18]  John Shalf,et al.  Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[19]  G. Bruce Berriman,et al.  The Application of Cloud Computing to Astronomy: A Study of Cost and Performance , 2010, 2010 Sixth IEEE International Conference on e-Science Workshops.

[20]  Charilaos Christopoulos,et al.  Transcoder architectures for video coding , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[21]  Gregory A. Koenig,et al.  Utility Functions and Resource Management in an Oversubscribed Heterogeneous Computing Environment , 2015, IEEE Transactions on Computers.

[22]  Rajkumar Buyya,et al.  Aneka: a Software Platform for .NET based Cloud Computing , 2009, High Performance Computing Workshop.

[23]  Rajkumar Buyya,et al.  Cost of Virtual Machine Live Migration in Clouds: A Performance Evaluation , 2009, CloudCom.

[24]  William May,et al.  HTTP Live Streaming , 2017, RFC.

[25]  Frédéric Desprez,et al.  Image Transfer and Storage Cost Aware Brokering Strategies for Multiple Clouds , 2014, 2014 IEEE 7th International Conference on Cloud Computing.

[26]  Sébastien Lafond,et al.  Analysis of video segmentation for spatial resolution reduction video transcoding , 2011, 2011 International Symposium on Intelligent Signal Processing and Communications Systems (ISPACS).

[27]  Luca De Cicco,et al.  Feedback control for adaptive live video streaming , 2011, MMSys.

[28]  Rajkumar Buyya,et al.  CVSS: A Cost-Efficient and QoS-Aware Video Streaming Using Cloud Services , 2016, 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid).

[29]  Sébastien Lafond,et al.  Video transcoding time prediction for proactive load balancing , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[30]  Fabrice Huet,et al.  Adaptive Fault Tolerance in Real Time Cloud Computing , 2011, 2011 IEEE World Congress on Services.

[31]  José Antonio Lozano,et al.  A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments , 2014, Journal of Grid Computing.

[32]  Bruce M. Maggs,et al.  Globally Distributed Content Delivery , 2002, IEEE Internet Comput..

[33]  Amir H. Payberah,et al.  CLive: Cloud-assisted P2P live streaming , 2012, 2012 IEEE 12th International Conference on Peer-to-Peer Computing (P2P).

[34]  Jiangchuan Liu,et al.  Understanding the Characteristics of Internet Short Video Sharing: A YouTube-Based Measurement Study , 2013, IEEE Transactions on Multimedia.

[35]  Rajkumar Buyya,et al.  High-Performance Cloud Computing: A View of Scientific Applications , 2009, 2009 10th International Symposium on Pervasive Systems, Algorithms, and Networks.

[36]  Magdy Bayoumi,et al.  Cloud-Based Video Streaming for Energy-and Compute-Limited Thin Clients , 2015 .

[37]  Sébastien Lafond,et al.  A Computation and Storage Trade-off Strategy for Cost-Efficient Video Transcoding in the Cloud , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[38]  John G. Apostolopoulos,et al.  On multiple description streaming with content delivery networks , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[39]  Baochun Li,et al.  Quality-assured cloud bandwidth auto-scaling for video-on-demand applications , 2012, 2012 Proceedings IEEE INFOCOM.

[40]  Ming-Ting Sun,et al.  Motion re-estimation for HDTV to SDTV transcoding , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[41]  Terry L. Kastens,et al.  Image masking for crop yield forecasting using AVHRR NDVI time series imagery , 2005 .

[42]  Yunhao Liu,et al.  AnySee: Peer-to-Peer Live Streaming , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[43]  Jiafu Wan,et al.  Cloud-assisted real-time transrating for http live streaming , 2013, IEEE Wireless Communications.

[44]  Yu Sun,et al.  Video transcoding: an overview of various techniques and research issues , 2005, IEEE Transactions on Multimedia.

[45]  Helen J. Wang,et al.  SecondNet: a data center network virtualization architecture with bandwidth guarantees , 2010, CoNEXT.

[46]  Anthony A. Maciejewski,et al.  Characterizing heterogeneous computing environments using singular value decomposition , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW).

[47]  J. Marchal.  Cours d'economie politique , 1950 .

[48]  Masahiro Shibata,et al.  Video on demand , 1996, Signal Process. Image Commun..

[49]  Inderveer Chana,et al.  Fault Tolerance- Challenges, Techniques and Implementation in Cloud Computing , 2012 .

[50]  Shiguo Lian,et al.  Multimedia Content Encryption: Techniques and Applications , 2008 .

[51]  Xinfeng Zhang,et al.  Parallelizing video transcoding with load balancing on cloud computing , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[52]  Prashant J. Shenoy,et al.  GreenCache: augmenting off-the-grid cellular towers with multimedia caches , 2013, MMSys.

[53]  Thomas Stockhammer,et al.  Dynamic adaptive streaming over HTTP --: standards and design principles , 2011, MMSys.

[54]  Anthony A. Maciejewski,et al.  Dynamically mapping tasks with priorities and multiple deadlines in a heterogeneous environment , 2007, J. Parallel Distributed Comput..

[55]  Eduardo Peixoto,et al.  MPEG-2 to HEVC Video Transcoding With Content-Based Modeling , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[56]  Anthony A. Maciejewski,et al.  Stochastic-Based Robust Dynamic Resource Allocation in a Heterogeneous Computing System , 2009, 2009 International Conference on Parallel Processing.

[57]  Heiko Schwarz,et al.  Overview of the Scalable Video Coding Extension of the H.264/AVC Standard , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[58]  Magdy A. Bayoumi,et al.  Fast Motion Estimation System Using Dynamic Models for H.264/AVC Video Coding , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[59]  Guillaume Pierre,et al.  EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications , 2009, ICSOC/ServiceWave Workshops.

[60]  Naoki Wakamiya,et al.  High-Speed Distributed Video Transcoding for Multiple Rates and Formats , 2005, IEICE Trans. Inf. Syst..

[61]  Phuoc Tran-Gia,et al.  A Survey on Quality of Experience of HTTP Adaptive Streaming , 2015, IEEE Communications Surveys & Tutorials.

[62]  Shuyu Chen,et al.  A Performance Comparison of Resource Allocation Policies in Distributed Computing Environments with Random Failures , 2012 .

[63]  Lavanya Ramakrishnan,et al.  Performance and energy efficiency of big data applications in cloud environments: A Hadoop case study , 2014, J. Parallel Distributed Comput..

[64]  Ben Y. Zhao,et al.  Silverline: toward data confidentiality in storage-intensive cloud applications , 2011, SoCC.

[65]  Timothy Bisson,et al.  iDedup: latency-aware, inline data deduplication for primary storage , 2012, FAST.

[66]  Magdy A. Bayoumi,et al.  VLSC: Video Live Streaming Using Cloud Services , 2016, 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom).

[67]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[68]  Qinghua Zheng,et al.  A version-aware computation and storage trade-off strategy for multi-version VoD systems in the cloud , 2015, 2015 IEEE Symposium on Computers and Communication (ISCC).

[69]  Manish Parashar,et al.  Latency Performance of SOAP Implementations , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[70]  Oliver Werner,et al.  Requantization for transcoding of MPEG-2 intraframes , 1999, IEEE Trans. Image Process..

[71]  Benjamin Farley,et al.  More for your money: exploiting performance heterogeneity in public clouds , 2012, SoCC '12.

[72]  Anthony A. Maciejewski,et al.  Stochastic robustness metric and its use for static resource allocations , 2008, J. Parallel Distributed Comput..

[73]  Ion Stoica,et al.  HTTP as the narrow waist of the future internet , 2010, Hotnets-IX.

[74]  Alexandru Iosup,et al.  A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing , 2009, CloudComp.

[75]  Sabela Ramos,et al.  General‐purpose computation on GPUs for high performance cloud computing , 2013, Concurr. Comput. Pract. Exp..

[76]  Anthony A. Maciejewski,et al.  Characterizing Task-Machine Affinity in Heterogeneous Computing Environments , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[77]  J. Lilius,et al.  Stream-Based Admission Control and Scheduling for Video Transcoding in Cloud Computing , 2013, 2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing.

[78]  Magdy A. Bayoumi,et al.  A Low Complexity Inter Mode Decision for MPEG-2 to H.264/AVC Video Transcoding in Mobile Environments , 2009, 2009 11th IEEE International Symposium on Multimedia.

[79]  Myoungjin Kim,et al.  Towards Efficient Design and Implementation of a Hadoop-based Distributed Video Transcoding System in Cloud Computing Environment , 2013 .

[80]  Randy H. Katz,et al.  Heterogeneity-Aware Resource Allocation and Scheduling in the Cloud , 2011, HotCloud.

[81]  A. Zahariev.  Google App Engine , 2009 .

[82]  Peter Fedor,et al.  A tribute to Claude Shannon (1916-2001) and a plea for more rigorous use of species richness, species diversity and the 'Shannon-Wiener' Index , 2003 .