No-Reference Stereoscopic Video Quality Assessment Algorithm Using Joint Motion and Depth Statistics

We propose a supervised no-reference (NR) quality assessment algorithm for assessing the perceptual quality of natural stereoscopic (S3D) videos. We empirically model the joint statistics of motion and depth subband coefficients of an S3D video frame using a Bivaraite Generalized Gaussian Distribution (BGGD). We compute the BGGD model parameters (α, β) to estimate the statistical dependency strength and show the features are quality discriminative. We compute the popular 2D NR image quality assessment (IQA) model NIQE on a frame-by-frame basis for both views to estimate the spatial quality. The frame-level BGGD features and spatial features are consolidated and used with the corresponding S3D videos difference mean opinion score (DMOS) labels for supervised learning using support vector regression (SVR). The overall quality of an S3D video is computed by averaging the frame-level quality predictions of the constituent video frames. The proposed algorithm, dubbed Video QUality Evaluation using MOtion and DEpth Statistics (VQUEMODES) is shown to outperform the state-of-the-art methods when evaluated over the IRCCYN and LFOVIA S3D subjective quality assessment databases.

[1]  Alan C. Bovik,et al.  No-Reference Quality Assessment of Natural Stereopairs , 2013, IEEE Transactions on Image Processing.

[2]  David Mumford,et al.  Statistics of range images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Do-Kyoung Kwon,et al.  Full-reference quality assessment of stereopairs accounting for rivalry , 2013, Signal Process. Image Commun..

[4]  Panos Nasiopoulos,et al.  An efficient human visual system based quality metric for 3D video , 2015, Multimedia Tools and Applications.

[5]  Bernhard Schölkopf,et al.  Comparing support vector machines with Gaussian kernels to radial basis function classifiers , 1997, IEEE Trans. Signal Process..

[6]  Damon M. Chandler,et al.  A spatiotemporal most-apparent-distortion model for video quality assessment , 2011, 2011 18th IEEE International Conference on Image Processing.

[7]  Jonathan Burton,et al.  RMIT3DV: Pre-announcement of a creative commons uncompressed HD 3D video database , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[8]  Munchurl Kim,et al.  A perceptual quality assessment metric using temporal complexity and disparity information for stereoscopic video , 2011, 2011 18th IEEE International Conference on Image Processing.

[9]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[10]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[11]  H. Komatsu,et al.  Disparity sensitivity of neurons in monkey extrastriate area MST , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  Ghassan Al-Regib,et al.  A no-reference quality measure for DIBR-based 3D videos , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[13]  Karen O. Egiazarian,et al.  Validation of a new full reference metric for quality assessment of mobile 3DTV content , 2011, 2011 19th European Signal Processing Conference.

[14]  Baihua Li,et al.  A no-reference optical flow-based quality evaluator for stereoscopic videos in curvelet domain , 2017, Inf. Sci..

[15]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[16]  Siwei Ma,et al.  Stereoscopic video quality assessment model based on spatial-temporal structural information , 2012, 2012 Visual Communications and Image Processing.

[17]  Alan C. Bovik,et al.  Natural scene statistics of color and range , 2011, 2011 18th IEEE International Conference on Image Processing.

[18]  Ahmet M. Kondoz,et al.  Perceptual Video Quality Metric for 3D video quality assessment , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[19]  Yanqing Li,et al.  No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics , 2017, 2017 2nd International Conference on Multimedia and Image Processing (ICMIP).

[20]  Mounir Kaaniche,et al.  No-reference stereo image quality assessment based on joint wavelet decomposition and statistical models , 2017, Signal Process. Image Commun..

[21]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality assessment for 3D video compression and transmission , 2011, IEEE Transactions on Consumer Electronics.

[22]  Jean-Yves Guillemaut,et al.  Stereoscopic Video Quality Assessment Using Binocular Energy , 2017, IEEE Journal of Selected Topics in Signal Processing.

[23]  Thomas Wiegand,et al.  3D video: acquisition, coding, and display , 2010, IEEE Transactions on Consumer Electronics.

[24]  Mei Yu,et al.  Binocular perception based reduced-reference stereo video quality assessment method , 2016, J. Vis. Commun. Image Represent..

[25]  Sumohana S. Channappayya,et al.  Full-Reference 3-D Video Quality Assessment Using Scene Component Statistical Dependencies , 2018, IEEE Signal Processing Letters.

[26]  Yang Liu,et al.  Disparity statistics in natural scenes. , 2008, Journal of vision.

[27]  Narciso García,et al.  NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[28]  Warnakulasuriya Anil Chandana Fernando,et al.  3D video assessment with Just Noticeable Difference in Depth evaluation , 2010, 2010 IEEE International Conference on Image Processing.

[29]  G. DeAngelis,et al.  Organization of Disparity-Selective Neurons in Macaque Area MT , 1999, The Journal of Neuroscience.

[30]  Michael R. Frater,et al.  No-reference quality assessment of 3D videos based on human visual perception , 2014, 2014 International Conference on 3D Imaging (IC3D).

[31]  Tai Sing Lee,et al.  Statistical correlations between two-dimensional images and three-dimensional structures in natural scenes. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[32]  Alan C. Bovik,et al.  No-Reference Image Quality Assessment in the Spatial Domain , 2012, IEEE Transactions on Image Processing.

[33]  Alan C. Bovik,et al.  Oriented Correlation Models of Distorted Natural Images With Application to Natural Stereopair Quality Evaluation , 2015, IEEE Transactions on Image Processing.

[34]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[35]  José Vinícius de Miranda Cardoso,et al.  Objective estimation of 3D video quality: A disparity-based weighting strategy , 2013, 2013 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[36]  Y. Horita,et al.  Spatio-temporal segmentation based continuous no-reference stereoscopic video quality prediction , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[37]  M. Jakubowski,et al.  Block-based motion estimation algorithms — a survey , 2013 .

[38]  Jean-Yves Tourneret,et al.  Parameter Estimation For Multivariate Generalized Gaussian Distributions , 2013, IEEE Transactions on Signal Processing.

[39]  Gabriel-Miro Muntean,et al.  Extended no reference objective Quality Metric for stereoscopic 3D video , 2015, 2015 IEEE International Conference on Communication Workshop (ICCW).

[40]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[41]  Alan C. Bovik,et al.  Luminance, disparity, and range statistics in 3D natural scenes , 2009, Electronic Imaging.

[42]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[43]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[44]  Patrick Le Callet,et al.  Towards a New Quality Metric for 3-D Synthesized View Assessment , 2011, IEEE Journal of Selected Topics in Signal Processing.

[45]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[46]  Joydeep Ghosh,et al.  Algorithmic assessment of 3D quality of experience for images and videos , 2011, 2011 Digital Signal Processing and Signal Processing Education Meeting (DSP/SPE).

[47]  Ahmet M. Kondoz,et al.  Quality Evaluation of Color Plus Depth Map-Based Stereoscopic Video , 2009, IEEE Journal of Selected Topics in Signal Processing.

[48]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[49]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[50]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[51]  Mei Yu,et al.  No reference stereo video quality assessment based on motion feature in tensor decomposition domain , 2018, J. Vis. Commun. Image Represent..

[52]  Sumohana S. Channappayya,et al.  A full reference stereoscopic video quality assessment metric , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[53]  Ahmet M. Kondoz,et al.  Analyzing perceptual attributes of 3d video , 2009, IEEE Transactions on Consumer Electronics.

[54]  Rana Fareed Ghani,et al.  Objective quality assessment of 3D stereoscopic video based on motion vectors and depth map features , 2015, 2015 7th Computer Science and Electronic Engineering Conference (CEEC).

[55]  D H HUBEL,et al.  RECEPTIVE FIELDS AND FUNCTIONAL ARCHITECTURE IN TWO NONSTRIATE VISUAL AREAS (18 AND 19) OF THE CAT. , 1965, Journal of neurophysiology.

[56]  Yang Liu,et al.  Dichotomy between luminance and disparity features at binocular fixations. , 2010, Journal of vision.

[57]  Lu Yu,et al.  A Spatio-Temporal Perceptual Quality Index Measuring Compression Distortions of Three-Dimensional Video , 2018, IEEE Signal Processing Letters.

[58]  Ahmet M. Kondoz,et al.  Toward an Impairment Metric for Stereoscopic Video: A Full-Reference Video Quality Metric to Assess Compressed Stereoscopic Video , 2013, IEEE Transactions on Image Processing.

[59]  Sumohana S. Channappayya,et al.  Subjective and Objective Study of the Relation Between 3D and 2D Views Based on Depth and Bitrate , 2017, SD&A.

[60]  Zhibo Chen,et al.  Blind Stereoscopic Video Quality Assessment: From Depth Perception to Overall Experience , 2018, IEEE Transactions on Image Processing.

[61]  Feng Qi,et al.  Stereoscopic video quality assessment based on visual attention and just-noticeable difference models , 2015, Signal, Image and Video Processing.

[62]  Mylène C. Q. Farias,et al.  A no-reference stereoscopic quality metric , 2015, Electronic Imaging.

[63]  Sumohana S. Channappayya,et al.  A novel sparsity-inspired blind image quality assessment algorithm , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[64]  A. Gotchev,et al.  A perceptual quality metric for high-definition stereoscopic 3D video , 2015, Electronic Imaging.

[65]  Alan C. Bovik,et al.  Statistical Modeling of 3-D Natural Scenes With Application to Bayesian Stereopsis , 2011, IEEE Transactions on Image Processing.

[66]  Sumohana S. Channappayya,et al.  Full-Reference Stereo Image Quality Assessment Using Natural Stereo Scene Statistics , 2015, IEEE Signal Processing Letters.

[67]  R A Young,et al.  The Gaussian derivative model for spatial vision: I. Retinal mechanisms. , 1988, Spatial vision.