3D reconstruction of dynamic scenes from two asynchronous video-streams

We present an algorithm for reconstruction of dynamic scenes from a video input of an asynchronous stereo-pair. Our method presumes that the timestamps of the frame acquisitions and the intrinsic parameters of the stereo cameras are known, as well as, that the extrinsic parameters can be estimated from static regions of the scene. It computes the 3D trajectory of each individual point by modeling its motion parameters. Our algorithm requires in addition to the two asynchronous images from the camera pair, two additional images from one of the cameras for its processing. Since dynamic scene elements cause wrong reconstructions, here we introduce a two stage process in which the directions of the possible motion components are estimated prior to the 3D pose estimation of the corresponding point.

[1]  Yaser Sheikh,et al.  3D Reconstruction of a Moving Point from a Series of 2D Projections , 2010, ECCV.

[2]  Lior Wolf,et al.  Homography Tensors: On Algebraic Entities that Represent Three Views of Static or Moving Planar Points , 2000, ECCV.

[3]  Mina Teicher,et al.  A General Framework for Trajectory Triangulation , 2004, Journal of Mathematical Imaging and Vision.

[4]  Takeo Kanade,et al.  Nonrigid Structure from Motion in Trajectory Space , 2008, NIPS.

[5]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[6]  Marcus A. Magnor,et al.  Making of Who Cares? HD Stereoscopic Free Viewpoint Video , 2011, 2011 Conference for Visual Media Production.

[7]  Darius Burschka,et al.  Dynamic 3D Mapping - Visual Estimation of Independent Motions for 3D Structures in Dynamic Environments , 2013, VISAPP.

[8]  Amnon Shashua,et al.  Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Hai Tao,et al.  Dynamic depth recovery from multiple synchronized video streams , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Marc Pollefeys,et al.  Visual-hull reconstruction from uncalibrated and unsynchronized video streams , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[11]  Hai Tao,et al.  Dynamic depth recovery from unsynchronized video streams , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  Darius Burschka,et al.  Rigid 3D geometry matching for grasping of known objects in cluttered scenes , 2012, Int. J. Robotics Res..

[13]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.