Parallax Geometry of Pairs of Points for 3D Scene Analysis

We present a geometric relationship between the image motion of pairs of points over multiple frames. This relationship is based on the parallax displacements of points with respect to an arbitrary planar surface, and does not involve epipolar geometry. A constraint is derived over two frames for any pair of points, relating their projective structure (with respect to the plane) based only on their image coordinates and their parallax displacements. Similarly, a 3D-rigidity constraint between pairs of points over multiple frames is derived. We show applications of these parallax-based constraints to solving three important problems in 3D scene analysis: (i) the recovery of 3D scene structure, (ii) the detection of moving objects in the presence of camera induced motion, and (iii) the synthesis of new camera views based on a given set of views. Moreover, we show that this approach can handle difficult situations for 3D scene analysis, e.g., where there is only a small set of parallax vectors, and in the presence of independently moving objects.

[1]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[2]  K. Hanna.  Direct multi-resolution estimation of ego-motion and structure from motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[3]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[4]  M. Shizawa,et al.  Principle of superposition: a common computational framework for analysis of multiple motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[5]  P. Burt,et al.  Mechanisms for isolating component patterns in the sequential analysis of multiple motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[6]  Patrick Bouthemy,et al.  Region-Based Tracking in an Image Sequence , 1992, ECCV.

[7]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[8]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[9]  Masahiko Shizawa,et al.  On Visual Ambiguities Due to Transparency in Motion and Stereo , 1992, ECCV.

[10]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Shmuel Peleg,et al.  A Three-Frame Algorithm for Estimating Two-Component Image Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[13]  Trevor Darrell,et al.  'Nulling' filters and the separation of transparent motions , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Andrew Zisserman,et al.  A Case Against Epipolar Geometry , 1993, Applications of Invariance in Computer Vision.

[15]  Roberto Cipolla,et al.  Robust Egomotion Estimation from Affine Motion Parallax , 1994, ECCV.

[16]  Harpreet S. Sawhney,et al.  3D geometry from planar parallax , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Michal Irani,et al.  Recovery of ego-motion using image stabilization , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Philip H. S. Torr,et al.  Stochastic Motion Clustering , 1994, ECCV.

[19]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 1994, ECCV.

[20]  Nassir Navab,et al.  Relative affine structure: theory and application to 3D reconstruction from perspective views , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Rakesh Kumar,et al.  Shape Recovery from Multiple Views: A Parallax Based Approach , 1994 .

[22]  Amnon Shashua,et al.  Projective Structure from Uncalibrated Images: Structure From Motion and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Amnon Shashua,et al.  Algebraic Functions For Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Andrew Zisserman,et al.  Motion Clustering using the Trilinear Constraint over Three Views , 1995 .

[25]  Michael Werman,et al.  Trilinearity of three perspective views and its associated tensor , 1995, Proceedings of IEEE International Conference on Computer Vision.

[26]  Andrew Zisserman,et al.  Robust detection of degenerate configurations for the fundamental matrix , 1995, Proceedings of IEEE International Conference on Computer Vision.

[27]  Michal Irani,et al.  Representation of scenes from collections of images , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[28]  Stefan Carlsson,et al.  Duality of reconstruction and positioning from projective views , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[29]  Olivier D. Faugeras,et al.  On the geometry and algebra of the point and line correspondences between N images , 1995, Proceedings of IEEE International Conference on Computer Vision.