Convex Relaxations for Consensus and Non-Minimal Problems in 3D Vision

In this paper, we formulate a generic non-minimal solver using the existing tools of Polynomials Optimization Problems (POP) from computational algebraic geometry. The proposed method exploits the well known Shor's or Lasserre's relaxations, whose theoretical aspects are also discussed. Notably, we further exploit the POP formulation of non-minimal solver also for the generic consensus maximization problems in 3D vision. Our framework is simple and straightforward to implement, which is also supported by three diverse applications in 3D vision, namely rigid body transformation estimation, Non-Rigid Structure-from-Motion (NRSfM), and camera autocalibration. In all three cases, both non-minimal and consensus maximization are tested, which are also compared against the state-of-the-art methods. Our results are competitive to the compared methods, and are also coherent with our theoretical analysis. The main contribution of this paper is the claim that a good approximate solution for many polynomial problems involved in 3D vision can be obtained using the existing theory of numerical computational algebra. This claim leads us to reason about why many relaxed methods in 3D vision behave so well? And also allows us to offer a generic relaxed solver in a rather straightforward way. We further show that the convex relaxation of these polynomials can easily be used for maximizing consensus in a deterministic manner. We support our claim using several experiments for aforementioned three diverse problems in 3D vision.

[1]  Jean B. Lasserre,et al.  Semidefinite Programming vs. LP Relaxations for Polynomial Programming , 2002, Math. Oper. Res..

[2]  Anders P. Eriksson,et al.  Guaranteed Outlier Removal with Mixed Integer Linear Programs , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  David J. Kriegman,et al.  Autocalibration via Rank-Constrained Estimation of the Absolute Quadric , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Andrea Fusiello,et al.  Practical Autocalibration , 2010, ECCV.

[5]  Luc Van Gool,et al.  Sampling Algebraic Varieties for Robust Camera Autocalibration , 2018, ECCV.

[6]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Adrien Bartoli,et al.  Generalizing the Prediction Sum of Squares Statistic and Formula, Application to Linear Fractional Image Warp and Surface Fitting , 2017, International Journal of Computer Vision.

[8]  Luca Carlone,et al.  Modeling Perceptual Aliasing in SLAM via Discrete–Continuous Graphical Models , 2018, IEEE Robotics and Automation Letters.

[9]  J. Gallier.  Quadratic Optimization Problems , 2020, Linear Algebra and Optimization with Applications to Machine Learning.

[10]  Hongdong Li,et al.  Consensus set maximization with guaranteed global optimality for robust geometry estimation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[11]  Anders P. Eriksson,et al.  Efficient Globally Optimal Consensus Maximisation with Tree Search , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jiaolong Yang,et al.  Optimal Essential Matrix Estimation via Inlier-Set Maximization , 2014, ECCV.

[13]  Anders P. Eriksson,et al.  Solving quadratically constrained geometrical problems using lagrangian duality , 2008, 2008 19th International Conference on Pattern Recognition.

[14]  David Fofi,et al.  Efficient Pruning LMI Conditions for Branch-and-Prune Rank and Chirality-Constrained Estimation of the Dual Absolute Quadric , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Amir Ali Ahmadi,et al.  Geometry of 3D Environments and Sum of Squares Polynomials , 2016, Robotics: Science and Systems.

[16]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[17]  David A. Forsyth,et al.  Capturing and animating occluded cloth , 2007, ACM Trans. Graph..

[18]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[19]  Didier Henrion,et al.  Globally Optimal Estimates for Geometric Reconstruction Problems , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[20]  Laurent Kneip,et al.  A Certifiably Globally Optimal Solution to the Non-minimal Relative Pose Problem , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Tat-Jun Chin,et al.  Robust Fitting in Computer Vision: Easy or Hard? , 2018, ECCV.

[22]  Adrien Bartoli,et al.  Isometric Non-Rigid Shape-from-Motion with Riemannian Geometry Solved in Linear Time , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Pascal Vasseur,et al.  Robust and Optimal Sum-of-Squares-Based Point-to-Plane Registration of Image Sets and Structured Scenes , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Luc Van Gool,et al.  Consensus Maximization with Linear Matrix Inequality Constraints , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  David J. Kriegman,et al.  Globally Optimal Affine and Metric Upgrades in Stratified Autocalibration , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[26]  John J. Leonard,et al.  SE-Sync: A certifiably correct algorithm for synchronization over the special Euclidean group , 2016, Int. J. Robotics Res..

[27]  Carl Olsson,et al.  Branch-and-Bound Methods for Euclidean Registration Problems , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Richard I. Hartley,et al.  Iterative Extensions of the Sturm/Triggs Algorithm: Convergence and Nonconvergence , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  John W. Chinneck,et al.  Feasibility and Infeasibility in Optimization:: Algorithms and Computational Methods , 2007 .

[30]  P. Parrilo.  Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization , 2000 .

[31]  Javier González,et al.  Convex Global 3D Registration with Lagrangian Duality , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Anders P. Eriksson,et al.  Rotation Averaging and Strong Duality , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Manolis I. A. Lourakis,et al.  Camera Self-Calibration Using the Singular Value Decomposition of the Fundamental Matrix: From Point Correspondences to 3D Measurements , 1999 .

[34]  Stephen P. Boyd,et al.  Semidefinite Programming Relaxations of Non-Convex Problems in Control and Combinatorial Optimization , 1997 .

[35]  Marc Pollefeys,et al.  Globally Optimal Inlier Set Maximization with Unknown Rotation and Focal Length , 2014, ECCV.

[36]  Hongdong Li,et al.  UPnP: An Optimal O(n) Solution to the Absolute Pose Problem with Universal Applicability , 2014, ECCV.

[37]  G. Alefeld,et al.  Interval analysis: theory and applications , 2000 .

[38]  Adrien Bartoli,et al.  Isometric Non-rigid Shape-from-Motion in Linear Time , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  V. Powers,et al.  An algorithm for sums of squares of real polynomials , 1998 .

[40]  Levente Hajder,et al.  A Minimal Solution for Two-View Focal-Length Estimation Using Two Affine Correspondences , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Huu Le,et al.  Deterministic consensus maximization with biconvex programming , 2018, ECCV.

[42]  Luc Van Gool,et al.  Consensus Maximization for Semantic Region Correspondences , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  J. Lasserre.  Convergent LMI relaxations for nonconvex quadratic programs , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).