Event-driven stitching for tile-based live 360 video streaming

360 video streaming is gaining popularity because of the new type of experience it creates. Tile-based approaches have been widely used in VoD 360 video streaming to save the network bandwidth. However, they cannot be extended to the case of live streaming because they assume the 360 videos stitched offline before streaming. Instead, stitching has to be done in real-time in live 360 video streaming. More importantly, the stitching speed as shown in our experiments is one order of magnitude lower than the network transmission speed, making stitching more of a deciding factor of the overall frame rate than the network transmission speed. In this paper, we design a stitching algorithm for tile-based live 360 video streaming that adapts stitching quality to make the best use of the timing budget. There are two main challenges. First, existing tile-based approaches do not consider various semantic information in different scenarios. Second, the decision of tiling schemes for tile-based stitching is non-trivial. To solve the above two challenges, we present an event-driven stitching algorithm for tile-based 360 video live streaming, which consists of such an event-driven model to abstract various semantic information as events and a tile actuator to make tiling scheme decisions. We implement a streaming system based on event-driven stitching called LiveTexture. To evaluate the proposed algorithm, we compare LiveTexture with other baseline systems and show that LiveTexture adapts well to various timing budgets by meeting 89.4% of the timing constraints. We also demonstrate that LiveTexture utilizes the timing budget more efficiently than others.

[1]  Miska M. Hannuksela,et al.  Viewport-Adaptive Encoding and Streaming of 360-Degree Video for Virtual Reality Applications , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[2]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[3]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[4]  覃政 Google Cardboard:伟大的搅局者 , 2014 .

[5]  Thomas Stockhammer,et al.  Dynamic adaptive streaming over HTTP --: standards and design principles , 2011, MMSys.

[6]  Songqing Chen,et al.  OpTile: Toward Optimal Tiling in 360-degree Video Streaming , 2017, ACM Multimedia.

[7]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[8]  Xinyu Zhang,et al.  POI360: Panoramic Mobile Video Telephony over LTE Cellular Networks , 2017, CoNEXT.

[9]  Paramvir Bahl,et al.  Live Video Analytics at Scale with Approximation and Delay-Tolerance , 2017, NSDI.

[10]  David Salomon,et al.  Transformations and projections in computer graphics , 2006 .

[11]  Zhenhua Li,et al.  A Measurement Study of Oculus 360 Degree Video Streaming , 2017, MMSys.

[12]  Afshin Taghavi Nasrabadi,et al.  Adaptive 360-Degree Video Streaming using Scalable Video Coding , 2017, ACM Multimedia.

[13]  Miska M. Hannuksela,et al.  HEVC-compliant Tile-based Streaming of Panoramic Video for Virtual Reality Applications , 2016, ACM Multimedia.

[14]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[15]  Bing-Yu Chen,et al.  High‐resolution 360 Video Foveated Stitching for Real‐time VR , 2017, Comput. Graph. Forum.

[16]  Zhimin Xu,et al.  360ProbDASH: Improving QoE of 360 Video Streaming Using Tile-based HTTP Adaptive Streaming , 2017, ACM Multimedia.

[17]  Gwendal Simon,et al.  Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming , 2017, ACM Multimedia.