A no-reference 3D virtual reality image quality assessment algorithm based on saliency statistics

In this work, we propose a supervised no-reference (NR) Image Quality Assessment (IQA) model for the objective evaluation of the perceptual quality of 3D virtual reality (VR) images. To achieve such practical algorithm, we first study the scene statistics of saliency maps of the individual left and right views of VR images, and empirically model these statistics with Univariate Generalized Gaussian Distribution (UGGD). We compute the UGGD model parameters at multi-scale and multi-orient steerable subband decomposition, and introduce these features as distortion discriminables. This is followed by the computation of the entropy and normalized root mean square scores of each subband and then these values are utilized as weights to pool the individual view features. We apply the popular 2D supervised BRISQUE model on the individual views to estimate the overall spatial quality of VR images. As the last step of the algorithm, the predicted saliency score and the spatial BRISQUE score are pooled to derive the final quality score of VR images. The performance of the proposed model is evaluated on the popular LIVE 3D VR IQA dataset. The results indicate robust and competitive performance against the off-the-shelf 2D full-reference and NR supervised algorithms.

[1]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[2]  Wilson S. Geisler,et al.  Image quality assessment based on a degradation model , 2000, IEEE Trans. Image Process..

[3]  Amine Bermak,et al.  Gradient magnitude similarity deviation on multiple scales for color image quality assessment , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Brian J. White,et al.  Residual Attention Guidance in Blindsight Monkeys Watching Complex Natural Scenes , 2012, Current Biology.

[5]  Yanqing Li,et al.  No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics , 2017, 2017 2nd International Conference on Multimedia and Image Processing (ICMIP).

[6]  Sumohana S. Channappayya,et al.  Full-Reference Stereo Image Quality Assessment Using Natural Stereo Scene Statistics , 2015, IEEE Signal Processing Letters.

[7]  David S. Doermann,et al.  Unsupervised feature learning framework for no-reference image quality assessment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9]  Bernd Girod,et al.  A Framework to Evaluate Omnidirectional Video Coding Schemes , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality.

[10]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Zulin Wang,et al.  Assessing Visual Quality of Omnidirectional Videos , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Chen Li,et al.  Bridge the Gap Between VQA and Human Behavior on Omnidirectional Video: A Large-Scale Dataset and a Deep Learning Model , 2018, ACM Multimedia.

[14]  Weihua Ye,et al.  Attentive Deep Stitching and Quality Assessment for 360$^{\circ }$ Omnidirectional Images , 2020, IEEE Journal of Selected Topics in Signal Processing.

[15]  Gordon Wetzstein,et al.  Saliency in VR: How Do People Explore Virtual Environments? , 2016, IEEE Transactions on Visualization and Computer Graphics.

[16]  David S. Doermann,et al.  No-Reference Image Quality Assessment Using Visual Codebooks , 2012, IEEE Transactions on Image Processing.

[17]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[18]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[19]  Balasubramanyam Appina,et al.  A ‘Complete Blind’ No-Reference Stereoscopic Image Quality Assessment Algorithm , 2020, 2020 International Conference on Signal Processing and Communications (SPCOM).

[20]  Sabine Süsstrunk,et al.  Saliency detection using maximum symmetric surround , 2010, 2010 IEEE International Conference on Image Processing.

[21]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[22]  Mohamed Cheriet,et al.  Mean Deviation Similarity Index: Efficient and Reliable Full-Reference Image Quality Evaluator , 2016, IEEE Access.

[23]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[24]  Sheila S. Hemami,et al.  VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images , 2007, IEEE Transactions on Image Processing.

[25]  Vladyslav Zakharchenko,et al.  Quality metric for spherical panoramic video , 2016, Optical Engineering + Applications.

[26]  Zulin Wang,et al.  Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Lu Yu,et al.  Weighted-to-Spherically-Uniform Quality Evaluation for Omnidirectional Video , 2017, IEEE Signal Processing Letters.

[28]  Ziad M Hafed,et al.  How is visual salience computed in the brain? Insights from behaviour, neurobiology and modelling , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[29]  Yong Man Ro,et al.  Deep Virtual Reality Image Quality Assessment With Human Perception Guider for Omnidirectional Image , 2020, IEEE Transactions on Circuits and Systems for Video Technology.