Model-free Consensus Maximization for Non-Rigid Shapes

Many computer vision methods use consensus maximization to relate measurements containing outliers with the correct transformation model. In the context of rigid shapes, this is typically done using Random Sampling and Consensus (RANSAC) by estimating an analytical model that agrees with the largest number of measurements (inliers). However, small parameter models may not be always available. In this paper, we formulate the model-free consensus maximization as an Integer Program in a graph using ‘rules’ on measurements. We then provide a method to solve it optimally using the Branch and Bound (BnB) paradigm. We focus its application on non-rigid shapes, where we apply the method to remove outlier 3D correspondences and achieve performance superior to the state of the art. Our method works with outlier ratio as high as 80%. We further derive a similar formulation for 3D template to image matching, achieving similar or better performance compared to the state of the art.

[1]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[2]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, ACM Trans. Graph..

[3]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[4]  Michael J. Black,et al.  SMPL: A Skinned Multi-Person Linear Model , 2023 .

[5]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[6]  Andrew W. Fitzgibbon,et al.  Metric Regression Forests for Correspondence Estimation , 2015, International Journal of Computer Vision.

[7]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Vincent Lepetit,et al.  Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation , 2008, International Journal of Computer Vision.

[10]  Anders P. Eriksson,et al.  Guaranteed Outlier Removal with Mixed Integer Linear Programs , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Leonidas J. Guibas,et al.  One Point Isometric Matching with the Heat Kernel , 2010, Comput. Graph. Forum.

[12]  Florent Brunet,et al.  Monocular template-based 3D surface reconstruction: Convex inextensible and nonconvex isometric methods , 2014, Comput. Vis. Image Underst..

[13]  Ron Kimmel,et al.  Spectral Generalized Multi-dimensional Scaling , 2013, International Journal of Computer Vision.

[14]  Huu Le,et al.  Conformal Surface Alignment with Optimal Möbius Search , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Hongdong Li,et al.  UPnP: An Optimal O(n) Solution to the Absolute Pose Problem with Universal Applicability , 2014, ECCV.

[16]  Hongdong Li,et al.  Consensus set maximization with guaranteed global optimality for robust geometry estimation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Daniel Pizarro-Perez,et al.  Shape-from-Template , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Sehoon Ha,et al.  Iterative Training of Dynamic Skills Inspired by Human Coaching Techniques , 2014, ACM Trans. Graph..

[19]  Pascal Fua,et al.  Template-Based Monocular 3D Shape Recovery Using Laplacian Meshes , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Luc Van Gool,et al.  Consensus Maximization with Linear Matrix Inequality Constraints , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[22]  David A. Forsyth,et al.  Capturing and animating occluded cloth , 2007, ACM Trans. Graph..

[23]  Pascal Vasseur,et al.  A Branch-and-Bound Approach to Correspondence and Grouping Problems , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Tat-Jun Chin,et al.  Robust Fitting in Computer Vision: Easy or Hard? , 2018, ECCV.

[25]  Adrien Bartoli,et al.  A Stable Analytical Framework for Isometric Shape-from-Template by Surface Integration , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Richard I. Hartley,et al.  Global Optimization through Rotation Space Search , 2009, International Journal of Computer Vision.

[27]  Johan Fredriksson,et al.  Fast and Reliable Two-View Translation Estimation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Michael J. Black,et al.  A Naturalistic Open Source Movie for Optical Flow Evaluation , 2012, ECCV.

[29]  Michael J. Black,et al.  FAUST: Dataset and Evaluation for 3D Mesh Registration , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Daniel Pizarro-Perez,et al.  Inextensible Non-Rigid Shape-from-Motion by Second-Order Cone Programming , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Bernt Schiele,et al.  Building statistical shape spaces for 3D human modeling , 2015, Pattern Recognit..

[32]  Pascal Fua,et al.  Linear Local Models for Monocular Reconstruction of Deformable Surfaces , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[34]  Adrien Bartoli,et al.  An Analysis of Errors in Graph-Based Keypoint Matching and Proposed Solutions , 2014, ECCV.

[35]  Daniel Cremers,et al.  Product Manifold Filter: Non-rigid Shape Correspondence via Kernel Density Estimation in the Product Space , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Stefan Hinz,et al.  MLPnP - A Real-Time Maximum Likelihood Solution to the Perspective-n-Point Problem , 2016, ArXiv.

[37]  Federico Tombari,et al.  SHOT: Unique signatures of histograms for surface and texture description , 2014, Comput. Vis. Image Underst..

[38]  Federico Tombari,et al.  Unique Signatures of Histograms for Local Surface Description , 2010, ECCV.

[39]  Leonidas J. Guibas,et al.  A concise and provably informative multi-scale signature based on heat diffusion , 2009 .

[40]  H. Seidel,et al.  Pattern-aware Deformation Using Sliding Dockers , 2011, SIGGRAPH 2011.

[41]  Hongdong Li,et al.  A practical algorithm for L triangulation with outliers , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Daniel Cremers,et al.  Efficient Deformable Shape Correspondence via Kernel Matching , 2017, 2017 International Conference on 3D Vision (3DV).

[43]  Marc Pollefeys,et al.  Globally Optimal Inlier Set Maximization with Unknown Rotation and Focal Length , 2014, ECCV.

[44]  Masatoshi Okutomi,et al.  Deterministically maximizing feasible subsystem for robust model fitting with unit norm constraint , 2011, CVPR 2011.

[45]  Garth P. McCormick,et al.  Computability of global solutions to factorable nonconvex programs: Part I — Convex underestimating problems , 1976, Math. Program..

[46]  Matthias Nießner,et al.  VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction , 2016, ECCV.

[47]  Adrien Bartoli,et al.  Inextensible Non-Rigid Structure-from-Motion by Second-Order Cone Programming , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Luc Van Gool,et al.  Stratified Self-Calibration with the Modulus Constraint , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Alexander M. Bronstein,et al.  Deep Functional Maps: Structured Prediction for Dense Shape Correspondence , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[50]  Tat-Jun Chin,et al.  The Maximum Consensus Problem: Recent Algorithmic Advances , 2017, Synthesis Lectures on Computer Vision.

[51]  Bodo Rosenhahn,et al.  3D Reconstruction of Human Motion from Monocular Image Sequences , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Pascal Fua,et al.  A constrained latent variable model , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Davide Scaramuzza,et al.  1-Point-RANSAC Structure from Motion for Vehicle-Mounted Cameras by Exploiting Non-holonomic Constraints , 2011, International Journal of Computer Vision.

[54]  Jean Ponce,et al.  Finding Matches in a Haystack: A Max-Pooling Strategy for Graph Matching in the Presence of Outliers , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Adrien Bartoli,et al.  Non-Rigid Shape-from-Motion for Isometric Surfaces using Infinitesimal Planarity , 2014, BMVC.

[56]  Daniel Pizarro-Perez,et al.  Feature-Based Deformable Surface Detection with Self-Occlusion Reasoning , 2011, International Journal of Computer Vision.

[57]  Daniel Cremers,et al.  A Combinatorial Solution to Non-Rigid 3D Shape-to-Image Matching , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).