Reconstructing 3D trajectories of independently moving objects using generic constraints

The 3D reconstruction of scenes containing independently moving objects from uncalibrated monocular sequences still poses serious challenges. Even if the background and the moving objects are rigid, each reconstruction is only known up to a certain scale, which results in a one-parameter family of possible, relative trajectories per moving object with respect to the background. In order to determine a realistic solution from this family of possible trajectories, this paper proposes to exploit the increased linear coupling between camera and object translations that tends to appear at false scales. An independence criterion is formulated in the sense of true object and camera motions being minimally correlated. The increased coupling at false scales can also lead to the destruction of special properties such as planarity, periodicity, etc. of the true object motion. This provides us with a second, 'non-accidentalness' criterion for the selection of the correct motion among the one-parameter family.

[1]  A. Edelman,et al.  Polynomial roots from companion matrix eigenvalues , 1995 .

[2]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Matthew Brand,et al.  Morphable 3D models from video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  M. Irani,et al.  Multi-body Segmentation : Revisiting Motion Consistency , 2002 .

[5]  Lior Wolf,et al.  On Projection Matrices and their Applications in Computer Vision , 2001, ICCV.

[6]  Lior Wolf,et al.  On Projection Matrices , 2001 .

[7]  Peter F. Sturm.  Structure and Motion for Dynamic Scenes - The Case of Points Moving in Planes , 2002, ECCV.

[8]  S. Shankar Sastry,et al.  Two-View Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix , 2002 .

[9]  Jitendra Malik,et al.  Motion segmentation and tracking using normalized cuts , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Mei Han,et al.  Multiple motion scene reconstruction from uncalibrated views , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  P. Torr.  Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[12]  T. Kanade,et al.  A multi-body factorization method for motion analysis , 1995, ICCV 1995.

[13]  René Vidal.  A Factorization Method for 3 D Multi-body Motion Estimation and Segmentation ∗ , 2002 .

[14]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[15]  Amnon Shashua,et al.  Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Lior Wolf,et al.  On Projection Matrices $$\mathcal{P}^k \to \mathcal{P}^2 ,k = 3,...,6, $$ and their Applications in Computer Vision , 2004, International Journal of Computer Vision.

[17]  Marc Pollefeys,et al.  Multiple view geometry , 2005 .

[18]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[19]  David G. Lowe,et al.  Three-Dimensional Object Recognition from Single Two-Dimensional Images , 1987, Artif. Intell..

[20]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[22]  A. Shashua,et al.  On projection matrices P/sup k//spl rarr/P/sup 2/, k=3,...,6, and their applications in computer vision , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[23]  Andrew W. Fitzgibbon,et al.  Multibody Structure and Motion: 3-D Reconstruction of Independently Moving Objects , 2000, ECCV.

[24]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[25]  Nuno Vasconcelos,et al.  Empirical Bayesian EM-based motion segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.