What Correspondences Reveal About Unknown Camera and Motion Models?

In two-view geometry, camera models and motion types are used as key knowledge along with the image point correspondences in order to solve several key problems of 3D vision. Problems such as Structure-from-Motion (SfM) and camera self-calibration are tackled under the assumptions of a specific camera projection model and motion type. However, these key assumptions may not be always justified, \ie., we may often know neither the camera model nor the motion type beforehand. In that context, one can extract only the point correspondences between images. From such correspondences, recovering two-view relationship --expressed by the unknown camera model and motion type-- remains to be an unsolved problem. In this paper, we tackle this problem in two steps. First, we propose a method that computes the correct two-view relationship in the presence of noise and outliers. Later, we study different possibilities to disambiguate the obtained relationships into camera model and motion type. By extensive experiments on both synthetic and real data, we verify our theory and assumptions in practical settings.

[1]  Richard I. Hartley,et al.  Global Optimization through Rotation Space Search , 2009, International Journal of Computer Vision.

[2]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[3]  Pascal Vasseur,et al.  A Branch-and-Bound Approach to Correspondence and Grouping Problems , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Masatoshi Okutomi,et al.  Deterministically maximizing feasible subsystem for robust model fitting with unit norm constraint , 2011, CVPR 2011.

[6]  Marc Pollefeys,et al.  A Minimal Case Solution to the Calibrated Relative Pose Problem for the Case of Two Known Orientation Angles , 2010, ECCV.

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  Å. Björck,et al.  Solution of Vandermonde Systems of Equations , 1970 .

[9]  Anders P. Eriksson,et al.  Efficient Globally Optimal Consensus Maximisation with Tree Search , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[11]  Hongdong Li,et al.  Consensus set maximization with guaranteed global optimality for robust geometry estimation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Garth P. McCormick,et al.  Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Bernd Sturmfels,et al.  Learning algebraic varieties from samples , 2018, Revista Matemática Complutense.

[15]  Davide Scaramuzza,et al.  1-Point-RANSAC Structure from Motion for Vehicle-Mounted Cameras by Exploiting Non-holonomic Constraints , 2011, International Journal of Computer Vision.

[16]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[17]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  David J. Kriegman,et al.  Autocalibration via Rank-Constrained Estimation of the Absolute Quadric , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[22]  Richard I. Hartley.  Self-Calibration from Multiple Views with a Rotating Camera , 1994, ECCV.

[23]  Andrew Zisserman,et al.  3D Motion recovery via affine Epipolar geometry , 1995, International Journal of Computer Vision.

[24]  Marc Pollefeys,et al.  Globally Optimal Inlier Set Maximization with Unknown Rotation and Focal Length , 2014, ECCV.

[25]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[26]  Luc Van Gool,et al.  Progressive Structure from Motion , 2018, ECCV.

[27]  Luc Van Gool,et al.  Consensus Maximization with Linear Matrix Inequality Constraints , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[29]  Anders P. Eriksson,et al.  Guaranteed Outlier Removal with Mixed Integer Linear Programs , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Wolfram Burgard,et al.  A benchmark for the evaluation of RGB-D SLAM systems , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.