Depth map post-processing for depth-image-based rendering: a user study

We analyse the impact of depth map post-processing techniques on the visual quality of stereo pairs that contain a novel view. To this end, we conduct a user study, in which we address (1) the effects of depth map post processing on the quality of stereo pairs that contain a novel view and (2) the question whether objective quality metrics are suitable for evaluating them. We generate depth maps of six stereo image pairs and apply six different post-processing techniques. The unprocessed and the post-processed depth maps are used to generate novel views. The original left views and the novel views form the stereo pairs that are evaluated in a paired comparison study. The obtained results are compared with the results delivered by the objective quality metrics. We show that post-processing depth maps significantly enhances the perceived quality of stereo pairs that include a novel view. We further observe that the correlation between subjective and objective quality is weak.

[1]  Alan C. Bovik,et al.  Image information and visual quality , 2006, IEEE Trans. Image Process..

[2]  Touradj Ebrahimi,et al.  Paired comparison-based subjective quality assessment of stereoscopic images , 2013, Multimedia Tools and Applications.

[3]  Carsten Rother,et al.  Fast cost-volume filtering for visual correspondence and beyond , 2011, CVPR 2011.

[4]  Fang Wei,et al.  Foreground-Object-Protected Depth Map Smoothing for DIBR , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[5]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[6]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[8]  M. Glickman Parameter Estimation in Large Dynamic Paired Comparison Experiments , 1999 .

[9]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[10]  Christian Schmid,et al.  A Matlab function to estimate choice model parameters from paired-comparison data , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[11]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[12]  Sheila S. Hemami,et al.  VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images , 2007, IEEE Transactions on Image Processing.

[13]  Patrick Le Callet,et al.  Reliability of 2D quality assessment methods for synthesized views evaluation in stereoscopic viewing conditions , 2012, 2012 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[14]  Zhou Wang,et al.  Multi-scale structural similarity for image quality assessment , 2003 .

[15]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[16]  A. Bovik,et al.  A universal image quality index , 2002, IEEE Signal Processing Letters.

[17]  Wilson S. Geisler,et al.  Image quality assessment based on a degradation model , 2000, IEEE Trans. Image Process..

[18]  Minh N. Do,et al.  Depth Video Enhancement Based on Weighted Mode Filtering , 2012, IEEE Transactions on Image Processing.

[19]  R. A. Bradley,et al.  Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons , 1952 .

[20]  Ahmet M. Kondoz,et al.  Quality Evaluation of Color Plus Depth Map-Based Stereoscopic Video , 2009, IEEE Journal of Selected Topics in Signal Processing.