Simultaneous two-view epipolar geometry estimation and motion segmentation by 4D tensor voting

We address the problem of simultaneous two-view epipolar geometry estimation and motion segmentation from nonstatic scenes. Given a set of noisy image pairs containing matches of n objects, we propose an unconventional, efficient, and robust method, 4D tensor voting, for estimating the unknown n epipolar geometries, and segmenting the static and motion matching pairs into n, independent motions. By considering the 4D isotropic and orthogonal joint image space, only two tensor voting passes are needed, and a very high noise to signal ratio (up to five) can be tolerated. Epipolar geometries corresponding to multiple, rigid motions are extracted in succession. Only two uncalibrated frames are needed, and no simplifying assumption (such as affine camera model or homographic model between images) other than the pin-hole camera model is made. Our novel approach consists of propagating a local geometric smoothness constraint in the 4D joint image space, followed by global consistency enforcement for extracting the fundamental matrices corresponding to independent motions. We have performed extensive experiments to compare our method with some representative algorithms to show that better performance on nonstatic scenes are achieved. Results on challenging data sets are presented.

[1]  Dana H. Ballard,et al.  Parameter Nets , 1984, Artif. Intell..

[2]  Mi-Suen Lee,et al.  N-Dimensional Tensor Voting and Application to Epipolar Geometry Estimation , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Richard I. Hartley,et al.  In defence of the 8-point algorithm , 1995, Proceedings of IEEE International Conference on Computer Vision.

[4]  Olivier D. Faugeras,et al.  The fundamental matrix: Theory, algorithms, and stability analysis , 2004, International Journal of Computer Vision.

[5]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[6]  S. Shankar Sastry,et al.  Optimal Motion Estimation from Multiview Normalized Epipolar Constraint , 2001, ICCV.

[7]  Philip H. S. Torr,et al.  Statistical detection of independent movement from a moving camera , 1993, Image Vis. Comput..

[8]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[9]  Rud Sturm,et al.  Das Problem der Projectivität und seine Anwendung auf die Flächen zweiten Grades , 1869 .

[10]  Philip H. S. Torr,et al.  Statistical detection of independent movement from a moving camera , 1993, Image Vis. Comput..

[11]  Ehud Rivlin,et al.  ROR: Rejection of Outliers by Rotations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  P. Anandan,et al.  Integrating Local Affine into Global Projective Images in the Joint Image Space , 2000, ECCV.

[13]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[15]  Gérard G. Medioni,et al.  Epipolar geometry estimation for non-static scenes by 4D tensor voting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[16]  Gérard G. Medioni,et al.  Perceptual Grouping from Motion Cues Using Tensor Voting in 4-D , 2002, ECCV.

[17]  Lior Wolf,et al.  Two-body segmentation from two perspective views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[19]  Mi-Suen Lee,et al.  A Computational Framework for Segmentation and Grouping , 2000 .

[20]  James Davis,et al.  Mosaics of scenes with moving objects , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[21]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[22]  Andrew Zisserman,et al.  Wide baseline stereo matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[23]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[24]  Chi-Keung Tang,et al.  A Computational Framework for Feature Extraction and Segmentation , 2000 .

[25]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[26]  Zhengyou Zhang,et al.  Determining the Epipolar Geometry and its Uncertainty: A Review , 1998, International Journal of Computer Vision.