Temporal enhancement of graph-based depth estimation method

This paper presents the temporal enhancement of the graph-based depth estimation method, designed for multiview systems with arbitrarily located cameras. The primary goal of the proposed enhancement is to increase the quality of estimated depth maps and simultaneously decrease the time of estimation. The method consists of two stages: the temporal enhancement of segmentation required in used depth estimation method, and the exploitation of depth information from the previous frame in the energy function minimization. Performed experiments show that for all tested sequences the quality of estimated depth maps was increased. Even if only one cycle of optimization is used in proposed method, the quality is higher than for unmodified method, apart from number of cycles. Therefore, use of proposed enhancement allows estimating depth of better quality even with 40% reduction of estimation time.

[1]  Jianjun Lei,et al.  Motion and Structure Information Based Adaptive Weighted Depth Video Estimation , 2015, IEEE Transactions on Broadcasting.

[2]  Frederik Zilly,et al.  Spatio-temporal consistent depth maps from multi-view video , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[3]  Peter Eisert,et al.  Real-time generation of multi-view video plus depth content using mixed narrow and wide baseline , 2014, J. Vis. Commun. Image Represent..

[4]  Krzysztof Wegner,et al.  Poznan University of Technology test multiview video sequences acquired with circular camera arrangement – “Poznan Team” and “Poznan Blocks” sequences , 2015 .

[5]  Marek Domanski,et al.  Graph-based multiview depth estimation using segmentation , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[6]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Feng Wu,et al.  Estimation of Virtual View Synthesis Distortion Toward Virtual View Position , 2016, IEEE Transactions on Image Processing.

[8]  Zixiang Xiong,et al.  A gradient-based approach for interference cancelation in systems with multiple Kinect cameras , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[9]  Toshiaki Fujii,et al.  FTV for 3-D Spatial Communication , 2012, Proceedings of the IEEE.

[10]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[11]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[12]  Takanori Senoh,et al.  New visual coding exploration in MPEG: Super-MultiView and Free Navigation in Free viewpoint TV , 2016, SD&A.

[13]  Krzysztof Wegner,et al.  Estimation of temporally-consistent depth maps from video with reduced noise , 2015, 2015 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[14]  King Ngi Ngan,et al.  Online Temporally Consistent Indoor Depth Video Enhancement via Static Structure , 2015, IEEE Transactions on Image Processing.

[15]  Gauthier Lafruit,et al.  Multi-view wide baseline depth estimation robust to sparse input sampling , 2016, 2016 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[16]  Hujun Bao,et al.  Spatio-Temporal Video Segmentation of Static Scenes and Its Applications , 2015, IEEE Transactions on Multimedia.

[17]  Patrick Ndjiki-Nya,et al.  Temporally consistent adaptive depth map preprocessing for view synthesis , 2013, 2013 Visual Communications and Image Processing (VCIP).

[18]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[19]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.