Multibody Structure and Motion: 3-D Reconstruction of Independently Moving Objects

This paper extends the recovery of structure and motion to image sequences with several independently moving objects. The motion, structure, and camera calibration are all a-priori unknown. The fundamental constraint that we introduce is that multiple motions must share the same camera parameters. Existing work on independent motions has not employed this constraint, and therefore has not gained over independent static-scene reconstructions. We show how this constraint leads to several new results in structure and motion recovery, where Euclidean reconstruction becomes possible in the multibody case, when it was underconstrained for a static scene. We show how to combine motions of high-relief, low-relief and planar objects. Additionally we show that structure and motion can be recovered from just 4 points in the uncalibrated, fixed camera, case. Experiments on real and synthetic imagery demonstrate the validity of the theory and the improvement in accuracy obtained using multibody analysis.

[1]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[2]  Luc Van Gool,et al.  The modulus constraint: a new constraint self-calibration , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[3]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[4]  Anders Heyden,et al.  Flexible calibration: minimal cases for auto-calibration , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Andrew Zisserman,et al.  Motion Clustering using the Trilinear Constraint over Three Views , 1995 .

[6]  Amnon Shashua,et al.  Trajectory triangulation of lines: reconstruction of a 3D point moving along a line from a monocular image sequence , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[7]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[8]  Bill Triggs,et al.  Autocalibration from Planar Scenes , 1998, ECCV.

[9]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[10]  Narendra Ahuja,et al.  Motion and Structure Factorization and Segmentation of Long Multiple Motion Image Sequences , 1992, ECCV.

[11]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[12]  Thierry Viéville,et al.  Using Singular Displacements for Uncalibrated Monocular Visual Systems , 1996, ECCV.

[13]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[14]  Michael Werman,et al.  Trajectory triangulation over conic section , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  Andrew Zisserman,et al.  Applications of Invariance in Computer Vision , 1993, Lecture Notes in Computer Science.

[16]  Philip H. S. Torr,et al.  Statistical detection of independent movement from a moving camera , 1993, Image Vis. Comput..

[17]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[19]  M. Pollefeys.  Self-calibration and metric 3d reconstruction from uncalibrated image sequences , 1999 .

[20]  Peter F. Sturm,et al.  Critical motion sequences for the self-calibration of cameras and stereo systems with variable focal length , 1999, Image Vis. Comput..

[21]  S. Bougnoux,et al.  From projective to Euclidean space under any practical situation, a criticism of self-calibration , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[22]  M. Brooks,et al.  Recovering unknown focal lengths in self-calibration: an essentially linear algorithm and degenerate configurations , 1996 .

[23]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[24]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[25]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[26]  Philip H. S. Torr,et al.  Statistical detection of independent movement from a moving camera , 1993, Image Vis. Comput..

[27]  Andrew W. Fitzgibbon,et al.  Maintaining multiple motion model hypotheses over many views to recover matching and structure , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[28]  Andrew Zisserman,et al.  Self-Calibration from Image Triplets , 1996, ECCV.

[29]  Reg G. Willson,et al.  What is the center of the image , 1994 .