Fast 3D Structure From Motion with Missing Points from Registration of Partial Reconstructions

Structure From Motion (SFM) technique is usually used for camera motion recovery and 3D shape estimation. But the major problem with SFM is the occlusion of feature points which lead to hallucinate them, and thus increase the computation time. In this paper, we propose a method for a 3D shape reconstruction from a video sequence based on registering multiple partial reconstructions (patches). The proposed method avoids relying on hallucination step, which means a reduction of computation time. A realistic 3D textured shape is provided using a texture mapping pipeline based on the recovered motion of the camera. Experimental results on both synthetic and real images show that the proposed method is more than 200 times (on average) faster than the classical SFM methods which need to hallucinate the occluded points.

[1]  Riccardo Mazzon.  Real-time structure from motion for monocular and stereo cameras , 2010, Melecon 2010 - 2010 15th IEEE Mediterranean Electrotechnical Conference.

[2]  Stéphane Christy,et al.  Euclidean Shape and Motion from Multiple Perspective Views by Affine Iterations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Larry H. Matthies,et al.  Kalman filter-based algorithms for estimating depth from image sequences , 1989, International Journal of Computer Vision.

[4]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Chia-Hsiang Wu,et al.  Three-Dimensional Modeling From Endoscopic Video Using Geometric Constraints Via Feature Positioning , 2007, IEEE Transactions on Biomedical Engineering.

[6]  David W. Jacobs,et al.  Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[8]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Sinisa Segvic,et al.  Online/Realtime Structure and Motion for General Camera Models , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[10]  Axel Pinz,et al.  Fast and Globally Convergent Structure and Motion Estimation for General Camera Models , 2006, BMVC.

[11]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[12]  Michel Dhome,et al.  Generic and real-time structure from motion using local bundle adjustment , 2009, Image Vis. Comput..

[13]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.