Recovering depth of background and foreground from a monocular video with camera motion

In this paper we propose a depth recovery approach for monocular videos with or without camera motion. By combining geometric information and moving object extraction, not only the depth of background but also the depth of foreground can be recovered. Furthermore, for cases involving complex camera motion such as fast moving, translating, vertical movement, we propose a novel global motion estimation (GME) method including effective outlier rejection to extract moving objects, and experiments demonstrate that the proposed GME method outperforms most of the state-of-the-art methods. The depth recovery approach we propose is tested on four video sequences with different camera movements. Experimental results show that our approach produces more accurate depth of both background and foreground than existing depth recovery methods.

[1]  Qionghai Dai,et al.  An efficient method for automatic stereoscopic conversion , 2008 .

[2]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Sebastiano Battiato,et al.  Depth map generation by image classification , 2004, IS&T/SPIE Electronic Imaging.

[4]  Frédéric Dufaux,et al.  Efficient, robust, and fast global motion estimation for video coding , 2000, IEEE Trans. Image Process..

[5]  Kwanghoon Sohn,et al.  A Stereoscopic Video Generation Method Using Stereoscopic Display Characterization and Motion Analysis , 2008, IEEE Transactions on Broadcasting.

[6]  Ce Liu,et al.  Depth Extraction from Video Using Non-parametric Sampling , 2012, ECCV.

[7]  Dimitrios Androutsos,et al.  Semi-automatic 2D to 3D image conversion using scale-space Random Walks and a graph cuts based depth prior , 2011, 2011 18th IEEE International Conference on Image Processing.

[8]  Ivan V. Bajic,et al.  Motion Vector Outlier Rejection Cascade for Global Motion Estimation , 2010, IEEE Signal Processing Letters.

[9]  Hujun Bao,et al.  Consistent depth maps recovery from a trinocular video sequence , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Hujun Bao,et al.  Consistent Depth Maps Recovery from a Video Sequence , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Peter Meer,et al.  Edge Detection with Embedded Confidence , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Liang-Gee Chen,et al.  A 2D-to-3D conversion system using edge information , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[14]  Liang Zhang,et al.  3D-TV Content Generation: 2D-to-3D Conversion , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[15]  Mohammed Ghazal,et al.  Robust Global Motion Estimation Oriented to Video Object Segmentation , 2008, IEEE Transactions on Image Processing.

[16]  Liang-Gee Chen,et al.  Depth Map Generation for 2D-to-3D Conversion by Short-Term Motion Assisted Color Segmentation , 2007, 2007 IEEE International Conference on Multimedia and Expo.