Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming

With the decreasing price of Head-Mounted Displays (HMDs), 360-degree videos are becoming popular. The streaming of such videos through the Internet with state of the art streaming architectures requires, to provide high immersion feeling, much more bandwidth than the median user's access bandwidth. To decrease the need for bandwidth consumption while providing high immersion to users, scientists and specialists proposed to prepare and encode 360-degree videos into quality-variable video versions and to implement viewport-adaptive streaming. Quality-variable versions are different versions of the same video with non-uniformly spread quality: there exists some so-called Quality Emphasized Regions (QERs). With viewport-adaptive streaming the client, based on head movement prediction, downloads the video version with the high quality region closer to where the user will watch. In this paper we propose a generic theoretical model to find out the optimal set of quality-variable video versions based on traces of head positions of users watching a 360-degree video. We propose extensions to adapt the model to popular quality-variable version implementations such as tiling and offset projection. We then solve a simplified version of the model with two quality levels and restricted shapes for the QER. With this simplified model, we show that an optimal set of four quality-variable video versions prepared by a streaming server, together with a perfect head movement prediction, allow for 45% bandwidth savings to display video with the same average quality as state of the art solutions or allows an increase of 102% of the displayed quality for the same bandwidth budget.

[1]  Thomas Schierl,et al.  Compressed domain video processing for tile based panoramic streaming using HEVC , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[2]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[3]  Mohammad Hosseini,et al.  Adaptive 360 VR Video Streaming Based on MPEG-DASH SRD , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[4]  Feng Qian,et al.  Optimizing 360 video delivery over cellular networks , 2016, ATC@MobiCom.

[5]  Lina J. Karam,et al.  Visual Saliency Prediction Using a Mixture of Deep Neural Networks , 2017, IEEE Transactions on Image Processing.

[6]  Minhua Zhou,et al.  An Overview of Tiles in HEVC , 2013, IEEE Journal of Selected Topics in Signal Processing.

[7]  Cyril Concolato,et al.  MPEG DASH SRD: spatial relationship description , 2016, MMSys.

[8]  Wei Tsang Ooi,et al.  Mixing Tile Resolutions in Tiled Video: A Perceptual Quality Assessment , 2014, NOSSDAV 2014.

[9]  Jianle Chen,et al.  Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding Standard , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Gwendal Simon,et al.  Viewport-adaptive navigable 360-degree video delivery , 2016, 2017 IEEE International Conference on Communications (ICC).

[11]  Miska M. Hannuksela,et al.  Viewport-Adaptive Encoding and Streaming of 360-Degree Video for Virtual Reality Applications , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[12]  Akio Kameda,et al.  Live streaming system for omnidirectional video , 2015, 2015 IEEE Virtual Reality (VR).

[13]  Bernd Girod,et al.  A Framework to Evaluate Omnidirectional Video Coding Schemes , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality.

[14]  BorjiAli,et al.  State-of-the-Art in Visual Attention Modeling , 2013 .

[15]  Carsten Griwodz,et al.  Tiling of panorama video for interactive virtual cameras: Overheads and potential bandwidth requirement reduction , 2015, 2015 Picture Coding Symposium (PCS).

[16]  Touradj Ebrahimi,et al.  Efficient video coding based on audio-visual focus of attention , 2011, J. Vis. Commun. Image Represent..

[17]  Gwendal Simon,et al.  360-Degree Video Head Movement Dataset , 2017, MMSys.

[18]  Cyril Concolato,et al.  Adaptive Streaming of HEVC Tiled Videos Using MPEG-DASH , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Miska M. Hannuksela,et al.  HEVC-compliant Tile-based Streaming of Panoramic Video for Virtual Reality Applications , 2016, ACM Multimedia.

[20]  Miska M. Hannuksela,et al.  Efficient Coding of 360-Degree Pseudo-Cylindrical Panoramic Video for Virtual Reality Applications , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[21]  Paolo Napoletano,et al.  Bayesian Integration of Face and Low-Level Cues for Foveated Video Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Cyril Concolato,et al.  Tiled-based adaptive streaming using MPEG-DASH , 2016, MMSys.

[23]  Thomas Schierl,et al.  Compressed Domain Video Processing for Tile Based Panoramic Streaming using SHVC , 2015, ImmersiveME@ACM Multimedia.

[24]  Miska M. Hannuksela,et al.  HEVC-compliant viewport-adaptive streaming of stereoscopic panoramic video , 2016, 2016 Picture Coding Symposium (PCS).

[25]  Zhenhua Li,et al.  A Measurement Study of Oculus 360 Degree Video Streaming , 2017, MMSys.