Subjective and Objective Video Quality Assessment of 3D Synthesized Views With Texture/Depth Compression Distortion

The quality assessment for synthesized video with texture/depth compression distortion is important for the design, optimization, and evaluation of the multi-view video plus depth (MVD)-based 3D video system. In this paper, the subjective and objective studies for synthesized view assessment are both conducted. First, a synthesized video quality database with texture/depth compression distortion is presented with subjective scores given by 56 subjects. The 140 videos are synthesized from ten MVD sequences with different texture/depth quantization combinations. Second, a full reference objective video quality assessment (VQA) method is proposed concerning about the annoying temporal flicker distortion and the change of spatio-temporal activity in the synthesized video. The proposed VQA algorithm has a good performance evaluated on the entire synthesized video quality database, and is particularly prominent on the subsets which have significant temporal flicker distortion induced by depth compression and view synthesis process.

[1]  Thomas Wiegand,et al.  3-D Video Representation Using Depth Maps , 2011, Proceedings of the IEEE.

[2]  Federica Battisti,et al.  A wavelet-based image quality metric for the assessment of 3D synthesized views , 2013, Electronic Imaging.

[3]  Weisi Lin,et al.  Estimating Just-Noticeable Distortion for Video , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Patrick Le Callet,et al.  An edge-based structural distortion indicator for the quality assessment of 3D synthesized views , 2012, 2012 Picture Coding Symposium.

[5]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[6]  CHRISTOPH FEHN,et al.  Interactive 3-DTV-Concepts and Key Technologies , 2006, Proceedings of the IEEE.

[7]  Ju Liu,et al.  A Novel Distortion Model and Lagrangian Multiplier for Depth Maps Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Zhenzhong Chen,et al.  Depth No-Synthesis-Error Model for View Synthesis in 3-D Video , 2011, IEEE Transactions on Image Processing.

[9]  Lu Fang,et al.  An Analytical Model for Synthesis Distortion Estimation in 3D Video , 2014, IEEE Transactions on Image Processing.

[10]  Manoranjan Paul,et al.  Just Noticeable Difference for Images With Decomposition Model for Separating Edge and Textured Regions , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Qionghai Dai,et al.  Joint Bit Allocation and Rate Control for Coding Multi-View Video Plus Depth Based 3D Video , 2013, IEEE Transactions on Multimedia.

[12]  Touradj Ebrahimi,et al.  A quality assessment protocol for free-viewpoint video sequences synthesized from decompressed depth data , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[13]  Rajiv Soundararajan,et al.  Study of Subjective and Objective Quality Assessment of Video , 2010, IEEE Transactions on Image Processing.

[14]  Ming-Ting Sun,et al.  Global motion estimation from coarsely sampled motion vector field and the applications , 2003, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Dongxiao Li,et al.  An Asymmetric Edge Adaptive Filter for Depth Generation and Hole Filling in 3DTV , 2010, IEEE Transactions on Broadcasting.

[16]  Gangyi Jiang,et al.  Regional Bit Allocation and Rate Distortion Optimization for Multiview Depth Video Coding With View Synthesis Distortion Model , 2013, IEEE Transactions on Image Processing.

[17]  Christine Guillemot,et al.  Perceptually-Friendly H.264/AVC Video Coding Based on Foveated Just-Noticeable-Distortion Model , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Michael R. Ibbotson,et al.  Effects of saccades on visual processing in primate MSTd , 2010, Vision Research.

[19]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[20]  Dong Tian,et al.  Boundary Artifact Reduction in View Synthesis of 3D Video: From Perspective of Texture-Depth Alignment , 2011, IEEE Transactions on Broadcasting.

[21]  Alan C. Bovik,et al.  Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos , 2010, IEEE Transactions on Image Processing.

[22]  Patrick Le Callet,et al.  Objective image quality assessment of 3D synthesized views , 2015, Signal Process. Image Commun..

[23]  C.-C. Jay Kuo,et al.  Efficient Multiview Depth Coding Optimization Based on Allowable Depth Distortion in View Synthesis , 2014, IEEE Transactions on Image Processing.

[24]  Erhan Ekmekcioglu,et al.  Depth Based Perceptual Quality Assessment for Synthesised Camera Viewpoints , 2010, UCMedia.

[25]  Patrick Le Callet,et al.  Towards a New Quality Metric for 3-D Synthesized View Assessment , 2011, IEEE Journal of Selected Topics in Signal Processing.

[26]  Alan C. Bovik,et al.  Multimodal Interactive Continuous Scoring of Subjective 3D Video Quality of Experience , 2014, IEEE Transactions on Multimedia.

[27]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[28]  Weisi Lin,et al.  Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics , 2013, IEEE Transactions on Image Processing.

[29]  Ahmet M. Kondoz,et al.  Toward an Impairment Metric for Stereoscopic Video: A Full-Reference Video Quality Metric to Assess Compressed Stereoscopic Video , 2013, IEEE Transactions on Image Processing.

[30]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[31]  M. Gibson,et al.  Beyond ANOVA: Basics of Applied Statistics. , 1986 .

[32]  Bing Zeng,et al.  A new three-step search algorithm for block motion estimation , 1994, IEEE Trans. Circuits Syst. Video Technol..

[33]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Heiko Schwarz,et al.  3D High-Efficiency Video Coding for Multi-View Video and Depth Data , 2013, IEEE Transactions on Image Processing.

[35]  Hsueh-Ming Hang,et al.  Quality assessment of 3D synthesized views with depth map distortion , 2013, 2013 Visual Communications and Image Processing (VCIP).

[36]  Patrick Le Callet,et al.  Considering Temporal Variations of Spatial Visual Distortions in Video Quality Assessment , 2009, IEEE Journal of Selected Topics in Signal Processing.

[37]  Houqiang Li,et al.  Multiview-Video-Plus-Depth Coding Based on the Advanced Video Coding Standard , 2013, IEEE Transactions on Image Processing.

[38]  Chang-Su Kim,et al.  Bit Allocation Algorithm With Novel View Synthesis Distortion Model for Multiview Video Plus Depth Coding , 2014, IEEE Transactions on Image Processing.

[39]  Wilson S. Geisler,et al.  Image quality assessment based on a degradation model , 2000, IEEE Trans. Image Process..

[40]  Margaret H. Pinson,et al.  A new standardized method for objectively measuring video quality , 2004, IEEE Transactions on Broadcasting.

[41]  Lu Yu,et al.  A perceptual metric for evaluating quality of synthesized sequences in 3DV system , 2010, Visual Communications and Image Processing.

[42]  C.-C. Jay Kuo,et al.  Rate-Distortion Optimized Rate Control for Depth Map-Based 3-D Video Coding , 2013, IEEE Transactions on Image Processing.

[43]  Alexander Raake,et al.  Evaluating Depth Perception of 3D Stereoscopic Videos , 2012, IEEE Journal of Selected Topics in Signal Processing.

[44]  Alan C. Bovik,et al.  Visual Importance Pooling for Image Quality Assessment , 2009, IEEE Journal of Selected Topics in Signal Processing.

[45]  Weisi Lin,et al.  Image Quality Assessment Based on Gradient Similarity , 2012, IEEE Transactions on Image Processing.

[46]  Ja-Ling Wu,et al.  Quality Assessment of Stereoscopic 3D Image Compression by Binocular Integration Behaviors , 2014, IEEE Transactions on Image Processing.