Crowdsourced Multi-View Live Video Streaming using Cloud Computing

Advances and commoditization of media generation devices enable capturing and sharing of any special event by multiple attendees. We propose a novel system to collect individual video streams (views) captured for the same event by multiple attendees, and combine them into multi-view videos, where viewers can watch the event from various angles, taking crowdsourced media streaming to a new immersive level. The proposed system is called Cloud-based Multi-View Crowdsourced Streaming (CMVCS), and it delivers multiple views of an event to viewers at the best possible video representation based on each viewer’s available bandwidth. The CMVCS is a complex system having many research challenges. In this paper, we focus on resource allocation of the CMVCS system. The objective of the study is to maximize the overall viewer satisfaction by allocating available resources to transcode views in an optimal set of representations, subject to computational and bandwidth constraints. We choose the video representation set to maximize QoE using Mixed Integer Programming. Moreover, we propose a Fairness-Based Representation Selection (FBRS) heuristic algorithm to solve the resource allocation problem efficiently. We compare our results with optimal and Top-N strategies. The simulation results demonstrate that FBRS generates near optimal results and outperforms the state-of-the-art Top-N policy, which is used by a large-scale system (Twitch).

[1]  Jacob Chakareski Transmission Policy Selection for Multi-View Content Delivery Over Bandwidth Constrained Channels , 2014, IEEE Transactions on Image Processing.

[2]  Minghua Chen,et al.  CALMS: Cloud-assisted live media streaming for globalized demands with time/region diversities , 2012, 2012 Proceedings IEEE INFOCOM.

[3]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[4]  Feng Wang,et al.  Crowdsourced live streaming over the cloud , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[5]  Mohamed M. Fouad,et al.  A Modified Multiview Video Streaming System Using 3-Tier Architecture , 2016 .

[6]  A. Murat Tekalp,et al.  Client-Driven Selective Streaming of Multiview Video for Interactive 3DTV , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Camilo C. Dorea,et al.  Attention-Weighted Texture and Depth Bit-Allocation in General-Geometry Free-Viewpoint Television , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Chao Mei,et al.  CloudStream: Delivering high-quality streaming videos through a cloud-based SVC proxy , 2011, 2011 Proceedings IEEE INFOCOM.

[9]  Srinivasan Seshan,et al.  Practical, Real-time Centralized Control for CDN-based Live Video Delivery , 2015, SIGCOMM.

[10]  Jacob Chakareski Uplink Scheduling of Visual Sensors: When View Popularity Matters , 2015, IEEE Transactions on Communications.

[11]  Mohamed Hefeeda,et al.  Energy-efficient multicasting of multiview 3D videos to mobile devices , 2012, TOMCCAP.

[12]  Jacob Chakareski,et al.  User-Action-Driven View and Rate Scalable Multiview Video Coding , 2013, IEEE Transactions on Image Processing.

[13]  Jacob Chakareski,et al.  Joint source and channel coding of view and rate scalable multi-view video , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[14]  Aggelos K. Katsaggelos,et al.  Maximizing user utility in video streaming applications , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[15]  Antonio Ortega,et al.  On Dependent Bit Allocation for Multiview Image Coding With Depth-Image-Based Rendering , 2011, IEEE Transactions on Image Processing.

[16]  Alberto Blanc,et al.  Optimal set of video representations in adaptive streaming , 2014, MMSys '14.

[17]  Takashi Watanabe,et al.  Multi-view video streaming with mobile cameras , 2014, 2014 IEEE Global Communications Conference.

[18]  P. Frossard,et al.  Interactive Multiview Video System With Low Complexity 2D Look Around at Decoder , 2013, IEEE Transactions on Multimedia.

[19]  Kashif Bilal,et al.  Impact of Multiple Video Representations in Live Streaming: A Cost, Bandwidth, and QoE Analysis , 2017, 2017 IEEE International Conference on Cloud Engineering (IC2E).

[20]  Gwendal Simon,et al.  YouTube live and Twitch: a tour of user-generated live streaming systems , 2015, MMSys.

[21]  Jacob Chakareski,et al.  View-Popularity-Driven Joint Source and Channel Coding of View and Rate Scalable Multi-View Video , 2015, IEEE Journal of Selected Topics in Signal Processing.

[22]  Jacob Chakareski,et al.  Wireless Streaming of Interactive Multi-View Video via Network Compression and Path Diversity , 2014, IEEE Transactions on Communications.

[23]  Alberto Blanc,et al.  Optimal Selection of Adaptive Streaming Representations , 2014, ACM Trans. Multim. Comput. Commun. Appl..

[24]  Mohamed Hefeeda,et al.  Adaptive streaming of interactive free viewpoint videos to heterogeneous clients , 2016, MMSys.

[25]  Mohamed Hefeeda,et al.  MASH: A rate adaptation algorithm for multiview video streaming over HTTP , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[26]  Antonio Ortega,et al.  Coding structure optimization for interactive multiview streaming in virtual world observation , 2008, 2008 IEEE 10th Workshop on Multimedia Signal Processing.

[27]  Bo Li,et al.  Coping With Heterogeneous Video Contributors and Viewers in Crowdsourced Live Streaming: A Cloud-Based Approach , 2016, IEEE Transactions on Multimedia.

[28]  Yusheng Ji,et al.  Multiple Description Coding and Recovery of Free Viewpoint Video for Wireless Multi-Path Streaming , 2015, IEEE Journal of Selected Topics in Signal Processing.

[29]  Song-Hai Zhang,et al.  A Survey on Multiview Video Synthesis and Editing , 2016 .