Impacts of Retina-Related Zones on Quality Perception of Omnidirectional Image

Virtual Reality (VR), which brings immersive experiences to viewers, has been gaining popularity in recent years. A key feature in VR systems is the use of omnidirectional content, which provides 360-degree views of scenes. In this work, we study the human quality perception of omnidirectional images, focusing on different zones surrounding the foveation point. For that purpose, an extensive subjective experiment is carried out to assess the perceptual quality of omnidirectional images with non-uniform quality. Through experimental results, the impacts of different zones are analyzed. Moreover, twenty-five objective quality metrics, including foveal quality metrics, are evaluated using our database. It is quantitatively shown that the zones corresponding to the fovea and parafovea of human eyes are extremely important for quality perception, while the impacts of the other zones corresponding to the perifovea and periphery are small. Besides, most of the investigated metrics are found to be not effective enough to reflect the quality perceived by viewers. Our database has been made available to the public.

[1]  Zhan Ma,et al.  Perceptual Quality Assessment of Immersive Images Considering Peripheral Vision Impact , 2018, ArXiv.

[2]  Vito Roberto.  Intelligent perceptual systems : new directions in computational perception , 1993 .

[3]  Touradj Ebrahimi,et al.  Subjective Quality Evaluation of Foveated Video Coding Using Audio-Visual Focus of Attention , 2011, IEEE Journal of Selected Topics in Signal Processing.

[4]  Zulin Wang,et al.  Assessing Visual Quality of Omnidirectional Videos , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[6]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[7]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[8]  Hongyu Li,et al.  SR-SIM: A fast and high performance IQA index based on spectral residual , 2012, 2012 19th IEEE International Conference on Image Processing.

[9]  Sanghoon Lee,et al.  Foveated video image analysis and compression gain measurements , 2000, 4th IEEE Southwest Symposium on Image Analysis and Interpretation.

[10]  Truong Cong Thang,et al.  A Perception-based Quality Metric for Omnidirectional Images , 2019, 2019 IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia).

[11]  Patrick Le Callet,et al.  A Dataset of Head and Eye Movements for 360 Degree Images , 2017, MMSys.

[12]  Weisi Lin,et al.  Analysis of Distortion Distribution for Pooling in Image Quality Prediction , 2016, IEEE Transactions on Broadcasting.

[13]  Wenjun Zhang,et al.  Using Free Energy Principle For Blind Image Quality Assessment , 2015, IEEE Transactions on Multimedia.

[14]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[15]  Patrick Le Callet,et al.  Which saliency weighting for omni directional image quality assessment? , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[16]  L. O. Harvey,et al.  Light-difference threshold and subjective brightness in the periphery of the visual field , 1973, Psychologische Forschung.

[17]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[18]  Krista A. Ehinger,et al.  Recognizing scene viewpoint using panoramic place representation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Gwendal Simon,et al.  Viewport-Driven Rate-Distortion Optimized 360º Video Streaming , 2018, 2018 IEEE International Conference on Communications (ICC).

[20]  I. Rentschler,et al.  Peripheral vision and pattern recognition: a review. , 2011, Journal of vision.

[21]  Xiongkuo Min,et al.  Perceptual Quality Assessment of Omnidirectional Images , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[22]  Cheng-Hsin Hsu,et al.  Is Foveated Rendering Perceivable in Virtual Reality?: Exploring the Efficiency and Consistency of Quality Assessment Methods , 2017, ACM Multimedia.

[23]  Desney S. Tan,et al.  Foveated 3D graphics , 2012, ACM Trans. Graph..

[24]  Weisi Lin,et al.  A Fast Reliable Image Quality Predictor by Fusing Micro- and Macro-Structures , 2017, IEEE Transactions on Industrial Electronics.

[25]  Aljoscha Smolic,et al.  Viewport-aware adaptive 360° video streaming using tiles for virtual reality , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[26]  Anh T. Pham,et al.  An Optimal Tile-Based Approach for Viewport-Adaptive 360-Degree Video Streaming , 2019, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[27]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[28]  A. Hendrickson,et al.  Organization of the Adult Primate Fovea , 2005 .

[29]  Mingkai Huang,et al.  Modeling the Perceptual Quality of Immersive Images Rendered on Head Mounted Displays: Resolution and Compression , 2018, IEEE Transactions on Image Processing.

[30]  Yao Wang,et al.  Q-STAR: A Perceptual Video Quality Model Considering Impact of Spatial, Temporal, and Amplitude Resolutions , 2012, IEEE Transactions on Image Processing.

[31]  Joohwan Kim,et al.  Latency Requirements for Foveated Rendering in Virtual Reality , 2017, ACM Trans. Appl. Percept..

[32]  Li Zhaoping,et al.  Understanding Vision: Theory, Models, and Data , 2014 .

[33]  Truong Cong Thang,et al.  Impact of delays on 360-degree video communications , 2017, 2017 TRON Symposium (TRONSHOW).

[34]  Anh T. Pham,et al.  A New Adaptation Approach for Viewport-adaptive 360-degree Video Streaming , 2017, 2017 IEEE International Symposium on Multimedia (ISM).

[35]  Lei Zhang,et al.  RFSIM: A feature based image quality assessment metric using Riesz transforms , 2010, 2010 IEEE International Conference on Image Processing.

[36]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[37]  Yong Man Ro,et al.  Deep Virtual Reality Image Quality Assessment With Human Perception Guider for Omnidirectional Image , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[38]  Anh T. Pham,et al.  A Study on Quality Metrics for 360 Video Communications , 2018, IEICE Trans. Inf. Syst..

[39]  Hojin Ha,et al.  Perceptually Unequal Packet Loss Protection by Weighting Saliency and Error Propagation , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Joseph C. Besharse,et al.  The retina and its disorders , 2011 .

[41]  Zhou Wang,et al.  Foveated wavelet image quality index , 2001, Optics + Photonics.

[42]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[43]  Wilson S. Geisler,et al.  Image quality assessment based on a degradation model , 2000, IEEE Trans. Image Process..

[44]  J. Edward Swan,et al.  Peripheral Stimulation and its Effect on Perceived Spatial Scale in Virtual Environments , 2013, IEEE Transactions on Visualization and Computer Graphics.

[45]  Marios S. Pattichis,et al.  Foveated video quality assessment , 2002, IEEE Trans. Multim..