Depth map upsampling and refinement for FTV systems

In free-viewpoint television, high-quality depth maps are substantial for a virtual view synthesis for free navigation purposes. In this paper, we propose a new method of depth map quality improvement and resolution increase. Our method is intended for low resolution depth maps acquired through estimation using multiview video. The proposed depth map quality improvement is based on segmentation of an acquired high-resolution image. In the estimated depth map, searching for outliers is performed in the neighborhood created from similar segments in the acquired view. Experimental results show high effectiveness of the presented method of depth maps refinement, especially for object edges.

[1]  Sebastian Thrun,et al.  Upsampling range data in dynamic environments , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Michael S. Brown,et al.  High-Quality Depth Map Upsampling and Completion for RGB-D Cameras , 2014, IEEE Transactions on Image Processing.

[3]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Krzysztof Wegner,et al.  New results in free-viewpoint television systems for horizontal virtual navigation , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[5]  Sebastian Thrun,et al.  A Noise‐aware Filter for Real‐time Depth Upsampling , 2008 .

[6]  Toshiaki Fujii,et al.  FTV for 3-D Spatial Communication , 2012, Proceedings of the IEEE.

[7]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Krzysztof Wegner,et al.  A practical approach to acquisition and processing of free viewpoint video , 2015, 2015 Picture Coding Symposium (PCS).

[9]  Dimitrios Androutsos,et al.  Depth guided image completion for structure and texture synthesis , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  David Tschumperlé,et al.  Superpixel-based depth map inpainting for RGB-D view synthesis , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[11]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Cheolkon Jung,et al.  Image-guided depth propagation using superpixel matching and adaptive autoregressive model , 2015, 2015 Visual Communications and Image Processing (VCIP).