An efficient approach for 2D to 3D video conversion based on structure from motion

With the popularity of 3D films, the conversion of existing 2D videos to 3D videos has attracted a wide interest in 3D content production. In this paper, we present an efficient approach for 2D to 3D video conversion based on structure from motion (SFM). The key contributions include a piece-wise SFM approach and a novel nonlinear depth warping considering the characteristics of stereoscopic 3D. The dense depth maps are generated and further refined with color segmentation. Experiments show that the proposed approach can yield more visually satisfactory results.

[1]  Bruno Lévy,et al.  Geometry-aware direction field processing , 2009, TOGS.

[2]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[3]  Yong Ju Jung,et al.  A novel 2D-to-3D conversion technique based on relative height-depth cue , 2009, Electronic Imaging.

[4]  Karni Wolowelsky,et al.  Automatic video to stereoscopic video conversion , 2005, IS&T/SPIE Electronic Imaging.

[5]  Yong Ju Jung,et al.  2D-to-3D conversion by using visual attention analysis , 2010, Electronic Imaging.

[6]  Marc Pollefeys,et al.  3D models from extended uncalibrated video sequences: addressing key-frame selection and projective drift , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[7]  J. Weber,et al.  On creating depth maps from monoscopic video using structure from motion , 2010 .

[8]  Jinchang Ren,et al.  Object-Based 2D-to-3D Video Conversion for Effective Stereoscopic Content Generation in 3D-TV Applications , 2011, IEEE Transactions on Broadcasting.

[9]  Ce Liu,et al.  Depth Extraction from Video Using Non-parametric Sampling , 2012, ECCV.

[10]  Aljoscha Smolic,et al.  Nonlinear disparity mapping for stereoscopic 3D , 2010, SIGGRAPH 2010.

[11]  Manbae Kim,et al.  2D to 3D stereoscopic conversion: depth-map estimation in a 2D single-view image , 2007, SPIE Optical Engineering + Applications.

[12]  Liang-Gee Chen,et al.  Block-based Vanishing Line and Vanishing Point Detection for 3D Scene Reconstruction , 2006, 2006 International Symposium on Intelligent Signal Processing and Communications.

[13]  Daniel Cohen-Or,et al.  Semi-automatic stereo extraction from video footage , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[14]  Qionghai Dai,et al.  A Novel Method for Semi-automatic 2D to 3D Video Conversion , 2008, 2008 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[15]  T. Sikora,et al.  From 2D- to Stereo- to Multi-view Video , 2007, 2007 3DTV Conference.

[16]  M. Gross,et al.  Nonlinear disparity mapping for stereoscopic 3D , 2010, ACM Trans. Graph..

[17]  Liang Zhang,et al.  Stereoscopic image generation based on depth images for 3D TV , 2005, IEEE Transactions on Broadcasting.

[18]  Hujun Bao,et al.  Stereoscopic Video Synthesis from a Monocular Video , 2007, IEEE Transactions on Visualization and Computer Graphics.

[19]  John R. Kender,et al.  Computational approaches to temporal sampling of video sequences , 2007, TOMCCAP.

[20]  Jenq-Neng Hwang,et al.  Fast Extraction of Objects of Interest from Images with Low Depth of Field , 2007 .

[21]  Matthew N. Dailey,et al.  Robust Key Frame Extraction for 3D Reconstruction from Video Streams , 2010, VISAPP.

[22]  Wen Gao,et al.  2D to 3D convertion based on edge defocus and segmentation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  Aljoscha Smolic,et al.  Depth image based compositing for stereo 3D , 2012, 2012 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[24]  Qionghai Dai,et al.  An efficient method for automatic stereoscopic conversion , 2008 .

[25]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[26]  Miao Liao,et al.  Video Stereolization: Combining Motion Analysis with User Interaction , 2012, IEEE Transactions on Visualization and Computer Graphics.

[27]  Luc Van Gool,et al.  Surviving Dominant Planes in Uncalibrated Structure and Motion Recovery , 2002, ECCV.

[28]  Andrew Zisserman,et al.  Multiple View Geometry , 1999 .

[29]  Cevahir Çigla,et al.  Real-time stereo matching algorithm for 3DTV , 2012, 2012 20th Signal Processing and Communications Applications Conference (SIU).

[30]  Sung-Il Chien,et al.  Real-Time 2D-to-3D Conversion for 3DTV using Time-Coherent Depth-Map Generation Method , 2014 .