On the performance of objective metrics for omnidirectional visual content

Omnidirectional image and video have gained popularity thanks to availability of capture and display devices for this type of content. Recent studies have assessed performance of objective metrics in predicting visual quality of omnidirectional content. These metrics, however, have not been rigorously validated by comparing their prediction results with ground-truth subjective scores. In this paper, we present a set of 360-degree images along with their subjective quality ratings. The set is composed of four contents represented in two geometric projections and compressed with three different codecs at four different bitrates. A range of objective quality metrics for each stimulus is then computed and compared to subjective scores. Statistical analysis is performed in order to assess performance of each objective quality metric in predicting subjective visual quality as perceived by human observers. Results show the estimated performance of the state-of-the-art objective metrics for omnidirectional visual content. Objective metrics specifically designed for 360-degree content do not outperform conventional methods designed for 2D images.

[1]  Touradj Ebrahimi,et al.  Testbed for subjective evaluation of omnidirectional visual content , 2016, 2016 Picture Coding Symposium (PCS).

[2]  Pascal Frossard,et al.  Geometry-driven quantization for omnidirectional image coding , 2016, 2016 Picture Coding Symposium (PCS).

[3]  Eckehard G. Steinbach,et al.  H.264 Based coding of omnidirectional Video , 2004, ICCVG.

[4]  Jisheng Li,et al.  Novel tile segmentation scheme for omnidirectional video , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[5]  Touradj Ebrahimi,et al.  Towards high efficiency video coding: Subjective evaluation of potential coding technologies , 2011, J. Vis. Commun. Image Represent..

[6]  Shree K. Nayar,et al.  Catadioptric omnidirectional camera , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Zehdreh Allen-Lafayette,et al.  Flattening the Earth, Two Thousand Years of Map Projections , 1998 .

[8]  Vladyslav Zakharchenko,et al.  Quality metric for spherical panoramic video , 2016, Optical Engineering + Applications.

[9]  Ned Greene,et al.  Environment Mapping and Other Applications of World Projections , 1986, IEEE Computer Graphics and Applications.

[10]  Mohammad Hosseini,et al.  Adaptive 360 VR Video Streaming: Divide and Conquer , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[11]  Pascal Frossard,et al.  Low bit-rate compression of omnidirectional images , 2009, 2009 Picture Coding Symposium.

[12]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[13]  Ross Cutler,et al.  Quality Assessment of Panorama Video for Videoconferencing Applications , 2005, 2005 IEEE 7th Workshop on Multimedia Signal Processing.

[14]  Jean-François Macq,et al.  Interactive omnidirectional video delivery: A bandwidth-effective approach , 2012, Bell Labs Technical Journal.

[15]  Sabine Süsstrunk,et al.  Measuring colorfulness in natural images , 2003, IS&T/SPIE Electronic Imaging.

[16]  Bernd Girod,et al.  A Framework to Evaluate Omnidirectional Video Coding Schemes , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality.