Optimal motion estimation from multiple images by normalized epipolar constraint

In this paper, we study the structure from motion problem as a constrained nonlinear least squares problem which minimizes the so called reprojection error subject to all constraints among multiple images. By converting this constrained optimization problem to an unconstrained one, we contend that multilinear constraints, when used for motion and structure estimation, need to be properly normalized, which makes them no longer tensors. We demonstrate this by using the bilinear epipolar constraints and show how they give rise to a multiview version of the (crossed) normalized epipolar constraint of two views [5]. Such a (crossed) normalized epipolar constraint serves as an optimal objective function for motion (and structure) estimation. This objective function further reveals certain statistic relationship between bilinear and trilinear constraints: Even the rectilinear motion can be correctlyestimated by the normalized epipolar constraint as a limit of generic cases, hence trilinear constraints are not really necessary. Since the so obtained objective function is defined naturally on a product of Stiefel manifolds, we show how to use geometric optimization techniques [2] to minimize such a function. Simulation and experimental results are presented to evaluate the proposed algorithm and verify our claims.

[1]  P. Perona,et al.  Motion estimation via dynamic vision , 1996, IEEE Trans. Autom. Control..

[2]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[3]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  C Tomasi,et al.  Shape and motion from image streams: a factorization method. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Amnon Shashua,et al.  Trilinearity in Visual Recognition by Alignment , 1994, ECCV.

[6]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[7]  R. Nelson,et al.  Visual Navigation , 1996 .

[8]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[9]  Bill Triggs,et al.  Matching constraints and the joint image , 1995, Proceedings of IEEE International Conference on Computer Vision.

[10]  David W. Murray,et al.  A unifying framework for structure and motion recovery from image sequences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[11]  Carlo Tomasi,et al.  Fast, robust, and consistent camera motion estimation , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[12]  A. Heyden,et al.  Algebraic properties of multilinear constraints , 1997 .

[13]  S. Shankar Sastry,et al.  Optimization Criteria, Sensitivity and Robustness of Motion and Structure Estimation , 1999, Workshop on Vision Algorithms.