Synthesis quality prediction model based on distortion intolerance

Free-viewpoint video system will provide viewers with freedom to navigate through the scene at different viewpoints. In the system, arbitrary viewpoints of videos are synthesized by the depth image-based rendering with multi-view plus depth videos. Despite the widespread of technologies for free-viewpoint video system, the field of quality assessment for the free-viewpoint video, especially the quality prediction of a synthesized image, has not yet been thoroughly investigated. This paper analyzes how distortions in color and depth images influence on the quality of a synthesized image. Then, an objective quality prediction model for a synthesized image is proposed based on the concept of intolerance of synthesis distortion. Experimental results show that the proposed model provides outstanding performance in predicting the quality of a synthesized image compared to other models.

[1]  Alan C. Bovik,et al.  Image information and visual quality , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[3]  Chaminda T. E. R. Hewage,et al.  Reduced-reference quality metric for 3D depth map transmission , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[4]  Patrick Le Callet,et al.  Can 3D synthesized views be reliably assessed through usual subjective and objective evaluation protocols? , 2011, 2011 18th IEEE International Conference on Image Processing.

[5]  Kwanghoon Sohn,et al.  Stereoscopic image quality metric based on binocular perception model , 2012, 2012 19th IEEE International Conference on Image Processing.

[6]  Kwanghoon Sohn,et al.  3D reconstruction from stereo images for interactions between real and virtual objects , 2005, Signal Process. Image Commun..

[7]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[8]  Toshiaki Fujii,et al.  View Generation with 3D Warping Using Depth Information for FTV , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[9]  Ghassan Al-Regib,et al.  A no-reference quality measure for DIBR-based 3D videos , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[10]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[11]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Woo-Shik Kim,et al.  3-D video coding system with enhanced rendered view quality , 2011 .

[13]  Masayuki Tanimoto,et al.  FTV: Free-viewpoint Television , 2006, Signal Process. Image Commun..

[14]  Martin Reisslein,et al.  Objective Video Quality Assessment Methods: A Classification, Review, and Performance Comparison , 2011, IEEE Transactions on Broadcasting.

[15]  Ghassan Al-Regib,et al.  3VQM: A vision-based quality measure for DIBR-based 3D videos , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[16]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[17]  Svitlana Zinger,et al.  iGLANCE: Transmission to medical high definition autostereoscopic displays , 2009, 2009 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[18]  Alan C. Bovik,et al.  Range image quality assessment by Structural Similarity , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Aljoscha Smolic,et al.  Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems , 2008, 2008 15th IEEE International Conference on Image Processing.

[20]  Patrick Le Callet,et al.  Towards a New Quality Metric for 3-D Synthesized View Assessment , 2011, IEEE Journal of Selected Topics in Signal Processing.

[21]  Pierre-Henri Conze,et al.  Objective view synthesis quality assessment , 2012, Electronic Imaging.

[22]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[23]  Patrick Le Callet,et al.  An edge-based structural distortion indicator for the quality assessment of 3D synthesized views , 2012, 2012 Picture Coding Symposium.

[24]  Kwanghoon Sohn,et al.  No-reference sharpness metric based on inherent sharpness , 2011 .

[25]  Kwanghoon Sohn,et al.  No-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception , 2014, IEEE Transactions on Circuits and Systems for Video Technology.