Structure from motion in dynamic scenes with multiple motions

In this study, an algorithm is proposed to solve the multi-body structure from motion (SfM) problem for the single camera case. The algorithm uses the epipolar criterion to segment the features belonging to independently moving objects. Once the features are segmented, corresponding objects are reconstructed individually by applying a sequential algorithm, which uses the previous structure to estimate the pose of the current frame. A tracker is utilized to increase the baseline and improve the F-matrix estimation, which is beneficial for both segmentation and 3D structure estimation. The experimental results on synthetic and real data demonstrate that our approach efficiently deals with the multi-body SfM problem.

[1]  S. Shankar Sastry,et al.  Two-View Multibody Structure from Motion , 2005, International Journal of Computer Vision.

[2]  Engin Tola.  Multi-view 3D Reconstruction of a Scene Containing Independently Moving Objects , 2005 .

[3]  Marc Pollefeys,et al.  Multiple view geometry , 2005 .

[4]  Rama Chellappa,et al.  Bayesian algorithms for simultaneous structure from motion estimation of multiple independently moving objects , 2005, IEEE Transactions on Image Processing.

[5]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[6]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[7]  Manolis I. A. Lourakis,et al.  The design and implementation of a generic sparse bundle adjustment software package based on the Le , 2004 .

[8]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Andrew W. Fitzgibbon,et al.  Multibody Structure and Motion: 3-D Reconstruction of Independently Moving Objects , 2000, ECCV.

[10]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[12]  P. Torr.  Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[13]  Manolis I. A. Lourakis,et al.  Independent 3D motion detection using residual parallax normal flow fields , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  P. Anandan,et al.  A unified approach to moving object detection in 2D and 3D scenes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[15]  A. Murat Tekalp,et al.  Digital Video Processing , 1995 .

[16]  Edward H. Adelson,et al.  Layered representations for vision and video , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[17]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.