Modeling the Perceptual Impact of Viewport Adaptation for Immersive Video

Immersive video offers the freedom to navigate inside the virtualized environment. Instead of streaming the entire bulky content, a viewport or field of view (FoV) adaptive streaming is preferred. We often stream the high-quality content within current viewport, but degraded-quality representation elsewhere, so as to reduce the network bandwidth consumption. We then could refine the quality when focusing to a new FoV. Therefore, in this work, we have attempted to model the perceptual response of the quality variations (through adapting the quantization and spatial resolution) with respect to the refinement duration, and reach at a product of two closed-form exponential functions that well explain the joint quantization and resolution induced quality impact. Analytical model is also cross-validated using another set of data with both Pearson and Spearman's rank rank correlations over 0.98. Our work would be devised to guide the bandwidth-quality optimized immersive video streaming.

[1]  Mingkai Huang,et al.  Modeling the Perceptual Quality of Immersive Images Rendered on Head Mounted Displays: Resolution and Compression , 2018, IEEE Transactions on Image Processing.

[2]  Touradj Ebrahimi,et al.  A simple method to obtain visual attention data in head mounted virtual reality , 2017, 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[3]  Wen Gao,et al.  A Study on the Quantization Scheme in H.264/AVC and Its Application to Rate Control , 2004, PCM.

[4]  Yao Wang,et al.  Perceptual video quality assessment on a mobile platform considering both spatial resolution and quantization artifacts , 2010, 2010 18th International Packet Video Workshop.

[5]  Jie Wu,et al.  Modeling the impact of spatial resolutions on perceptual quality of immersive image/video , 2016, 2016 International Conference on 3D Imaging (IC3D).

[6]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[7]  Feng Li,et al.  Ultra Wide View Based Panoramic VR Streaming , 2017, VR/AR Network@SIGCOMM.

[8]  Shaowei Xie,et al.  Dynamic VR live streaming over MMT , 2017, 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[9]  Zhan Ma,et al.  Modeling of Rate and Perceptual Quality of Compressed Video as Functions of Frame Rate and Quantization Stepsize and Its Applications , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Gwendal Simon,et al.  Viewport-adaptive navigable 360-degree video delivery , 2016, 2017 IEEE International Conference on Communications (ICC).

[11]  Yong Liu,et al.  Prioritized Buffer Control in Two-tier 360 Video Streaming , 2017, VR/AR Network@SIGCOMM.

[12]  Christian Timmerer,et al.  Towards Bandwidth Efficient Adaptive Streaming of Omnidirectional Video over HTTP: Design, Implementation, and Evaluation , 2017, MMSys.

[13]  Gordon Wetzstein,et al.  Saliency in VR: How Do People Explore Virtual Environments? , 2016, IEEE Transactions on Visualization and Computer Graphics.

[14]  Zhan Ma,et al.  Optimization of spatial, temporal and amplitude resolution for rate-constrained video coding and scalable video adaptation , 2012, 2012 19th IEEE International Conference on Image Processing.