Tensor Oriented No-Reference Light Field Image Quality Assessment

Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media. However, due to the inherent high dimensional characteristics of LFI, the LFI quality assessment turns into a multi-dimensional problem that requires consideration of the quality degradation in both spatial and angular dimensions. Therefore, we propose a novel Tensor oriented No-reference Light Field image Quality evaluator (Tensor-NLFQ) based on tensor theory. Specifically, since the LFI is regarded as a low-rank 4D tensor, the principal components of four oriented sub-aperture view stacks are obtained via Tucker decomposition. Then, the Principal Component Spatial Characteristic (PCSC) is designed to measure the spatial-dimensional quality of LFI considering its global naturalness and local frequency properties. Finally, the Tensor Angular Variation Index (TAVI) is proposed to measure angular consistency quality by analyzing the structural similarity distribution between the first principal component and each view in the view stack. Extensive experimental results on four publicly available LFI quality databases demonstrate that the proposed Tensor-NLFQ model outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.

[1]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[2]  Alan C. Bovik,et al.  Bivariate statistical modeling of color and range in natural scenes , 2014, Electronic Imaging.

[3]  Ghassan Al-Regib,et al.  CSV: Image quality assessment based on color, structure, and visual system , 2016, Signal Process. Image Commun..

[4]  Zhibo Chen,et al.  Blind Stereoscopic Video Quality Assessment: From Depth Perception to Overall Experience. , 2018, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[5]  Federica Battisti,et al.  Towards the Perceptual Quality Evaluation of Compressed Light Field Images , 2017, IEEE Transactions on Broadcasting.

[6]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[7]  Hyun Gi Kim,et al.  Quantitative Analysis of the Effect of Iterative Reconstruction Using a Phantom: Determining the Appropriate Blending Percentage , 2015, Yonsei medical journal.

[8]  Wilson S. Geisler,et al.  Image quality assessment based on a degradation model , 2000, IEEE Trans. Image Process..

[9]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[10]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Abdul Rehman,et al.  Reduced-Reference Image Quality Assessment by Structural Similarity Estimation , 2012, IEEE Transactions on Image Processing.

[13]  Hao Jiang,et al.  New quality assessment method for dense light fields , 2018, SPIE/COS Photonics Asia.

[14]  Maria G. Martini,et al.  The Effect of Light Field Reconstruction and Angular Resolution Reduction on the Quality of Experience , 2016, 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[15]  Sheila S. Hemami,et al.  VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images , 2007, IEEE Transactions on Image Processing.

[16]  Konstantinos N. Plataniotis,et al.  Toward a No-Reference Image Quality Assessment Using Statistics of Perceptual Color Descriptors , 2016, IEEE Transactions on Image Processing.

[17]  Konstantinos N. Plataniotis,et al.  Towards a Full-Reference Quality Assessment for Color Images Using Directional Statistics , 2015, IEEE Transactions on Image Processing.

[18]  Dragan D. Kukolj,et al.  Multi–Scale Synthesized View Assessment Based on Morphological Pyramids , 2016 .

[19]  Jean-Yves Tourneret,et al.  Parameter Estimation For Multivariate Generalized Gaussian Distributions , 2013, IEEE Transactions on Signal Processing.

[20]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[21]  Zhibo Chen,et al.  Belif: Blind Quality Evaluator Of Light Field Image With Tensor Structure Variation Index , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[22]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[23]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[24]  Zhou Wang,et al.  Perceptual quality assessment of color images using adaptive signal representation , 2010, Electronic Imaging.

[25]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, SIGGRAPH 2011.

[26]  Patrick Le Callet,et al.  DIBR synthesized image quality assessment based on morphological wavelets , 2015, 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX).

[27]  Touradj Ebrahimi,et al.  Comparison and Evaluation of Light Field Image Coding Approaches , 2017, IEEE Journal of Selected Topics in Signal Processing.

[28]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[29]  Patrick Le Callet,et al.  DIBR synthesized image quality assessment based on morphological pyramids , 2015, 2015 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[30]  Wei Zhou,et al.  Perceptual Evaluation of Light Field Image , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[31]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[32]  Lei Zhang,et al.  Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index , 2013, IEEE Transactions on Image Processing.

[33]  Alan C. Bovik,et al.  Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality , 2011, IEEE Transactions on Image Processing.

[34]  Zhou Wang,et al.  Reduced- and No-Reference Image Quality Assessment , 2011, IEEE Signal Processing Magazine.

[35]  Guangming Shi,et al.  Perceptual Quality Metric With Internal Generative Mechanism , 2013, IEEE Transactions on Image Processing.

[36]  Federica Battisti,et al.  Reduced Reference Quality Assessment of Light Field Images , 2019, IEEE Transactions on Broadcasting.

[37]  Alberto Leon-Garcia,et al.  Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[38]  Patrick Le Callet,et al.  Objective image quality assessment of 3D synthesized views , 2015, Signal Process. Image Commun..

[39]  Alan C. Bovik,et al.  Perceptual quality prediction on authentically distorted images using a bag of features approach , 2016, Journal of vision.

[40]  Touradj Ebrahimi,et al.  VALID: Visual quality Assessment for Light field Images Dataset , 2018, 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).

[41]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[42]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[43]  Daniel Thalmann,et al.  Model-Based Referenceless Quality Metric of 3D Synthesized Images Using Local Image Description , 2018, IEEE Transactions on Image Processing.

[44]  Hans-Peter Seidel,et al.  Towards a Quality Metric for Dense Light Fields , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[46]  Christophe Charrier,et al.  Blind Image Quality Assessment: A Natural Scene Statistics Approach in the DCT Domain , 2012, IEEE Transactions on Image Processing.

[47]  Junhui Hou,et al.  Light Filed Image Quality Assessment by Local and Global Features of Epipolar Plane Image , 2018, 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM).

[48]  Weisi Lin,et al.  Perceptual visual quality metrics: A survey , 2011, J. Vis. Commun. Image Represent..

[49]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[50]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[51]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[52]  Qionghai Dai,et al.  Light Field Image Processing: An Overview , 2017, IEEE Journal of Selected Topics in Signal Processing.

[53]  Constantine Caramanis,et al.  Towards a Closed Form Second-Order Natural Scene Statistics Model , 2018, IEEE Transactions on Image Processing.

[54]  Wei Zhou,et al.  3D-HEVC visual quality assessment: Database and bitstream model , 2016, 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX).

[55]  Chong-Ho Choi,et al.  Incremental $N$ -Mode SVD for Large-Scale Multilinear Generative Models , 2014, IEEE Transactions on Image Processing.

[56]  Zhou Wang,et al.  Reduced-reference image quality assessment using a wavelet-domain natural image statistic model , 2005, IS&T/SPIE Electronic Imaging.

[57]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[58]  Alan Conrad Bovik,et al.  Binocular spatial activity and reverse saliency driven no-reference stereopair quality assessment , 2017, Signal Process. Image Commun..

[59]  Zhou Wang,et al.  Information Content Weighting for Perceptual Image Quality Assessment , 2011, IEEE Transactions on Image Processing.