Reconstruction of a Scene with Multiple Linearly Moving Objects

In this paper we describe an algorithm to recover the scene structure, the trajectories of the moving objects and the camera motion simultaneously given a monocular image sequence. The number of the moving objects is automatically detected without prior motion segmentation. Assuming that the objects are moving linearly with constant speeds, we propose a unified geometrical representation of the static scene and the moving objects. This representation enables the embedding of the motion constraints into the scene structure, which leads to a factorization-based algorithm. We also discuss solutions to the degenerate cases which can be automatically detected by the algorithm. Extension of the algorithm to weak perspective projections is presented as well. Experimental results on synthetic and real images show that the algorithm is reliable under noise.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  Amnon Shashua,et al.  Trajectory triangulation of lines: reconstruction of a 3D point moving along a line from a monocular image sequence , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Mei Han,et al.  Homography-Based 3D Scene Analysis of Video Sequences * , 1999 .

[4]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[5]  Daphna Weinshall,et al.  From Reference Frames to Reference Planes: Multi-View Parallax Geometry and Applications , 1998, ECCV.

[6]  Michal Irani,et al.  Recovery of Ego-Motion Using Region Alignment , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  P. Anandan,et al.  Direct recovery of shape from multiple views: a parallax based approach , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[8]  Lihi Zelnik-Manor,et al.  Multi-view subspace constraints on homographies , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[9]  Mei Han,et al.  Reconstruction of a Scene with Multiple Linearly Moving Objects , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Amnon Shashua,et al.  Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Lior Wolf,et al.  On Projection Matrices $$\mathcal{P}^k \to \mathcal{P}^2 ,k = 3,...,6, $$ and their Applications in Computer Vision , 2004, International Journal of Computer Vision.

[12]  Amnon Shashua,et al.  On the synthesis of dynamic scenes from reference views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[13]  P. Anandan,et al.  Direct Recovery of Planar-Parallax from Multiple Frames , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Charles R. Dyer,et al.  Interpolating view and scene motion by dynamic view morphing , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  Harpreet S. Sawhney,et al.  Independent motion detection in 3D scenes , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  T. Boult,et al.  Factorization-based segmentation of motions , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[17]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[19]  C. W. Gear,et al.  Multibody Grouping from Motion Images , 1998, International Journal of Computer Vision.

[20]  T. Kanade,et al.  The Factorization Method with Linear Motions , 1999 .

[21]  Lior Wolf,et al.  Homography Tensors: On Algebraic Entities that Represent Three Views of Static or Moving Planar Points , 2000, ECCV.

[22]  Philip H. S. Torr,et al.  Outlier detection and motion segmentation , 1993, Other Conferences.

[23]  Stéphane Christy,et al.  Euclidean Reconstruction: From Paraperspective to Perspective , 1996, ECCV.

[24]  Michal Irani,et al.  Multi-frame optical flow estimation using subspace constraints , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  Takeo Kanade,et al.  A Paraperspective Factorization Method for Shape and Motion Recovery , 1994, ECCV.

[26]  Rakesh Kumar,et al.  Shape Recovery from Multiple Views: A Parallax Based Approach , 1994 .

[27]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[28]  Michael Werman,et al.  Trajectory triangulation over conic section , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.