PL1P - Point-line Minimal Problems under Partial Visibility in Three Views

We present a complete classification of minimal problems for generic arrangements of points and lines in space observed partially by three calibrated perspective cameras when each line is incident to at most one point. This is a large class of interesting minimal problems that allows missing observations in images due to occlusions and missed detections. There is an infinite number of such minimal problems; however, we show that they can be reduced to 140616 equivalence classes by removing superfluous features and relabeling the cameras. We also introduce camera-minimal problems, which are practical for designing minimal solvers, and show how to pick a simplest camera-minimal problem for each minimal problem. This simplification results in 74575 equivalence classes. Only 76 of these were known; the rest are new. In order to identify problems that have potential for practical solving of image matching and 3D reconstruction, we present several smaller natural subfamilies of camera-minimal problems as well as compute solution counts for all camera-minimal problems which have less than 300 solutions for generic data.

[1]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Roland Siegwart,et al.  A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation , 2011, CVPR 2011.

[3]  Stepán Obdrzálek,et al.  Local affine frames for wide-baseline stereo , 2002, Object recognition supported by user interaction for service robots.

[4]  Zuzana Kukelova,et al.  Making Minimal Solvers for Absolute Pose Estimation Compact and Robust , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Torsten Sattler,et al.  Minimal Solvers for Generalized Pose and Scale Estimation from Two Rays and One Point , 2016, ECCV.

[6]  Jean Ponce,et al.  Trinocular Geometry Revisited , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  David Nister,et al.  Recent developments on direct relative orientation , 2006 .

[8]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Björn Johansson,et al.  Structure and Motion Estimation from Complex Features in Three Views , 2002, ICVGIP.

[11]  Matthew Trager.  Cameras, Shapes, and Contours: Geometric Models in Computer Vision. (Caméras, formes et contours: modèles géométriques en vision par ordinateur) , 2018 .

[12]  Richard I. Hartley,et al.  Lines and Points in Three Views and the Trifocal Tensor , 1997, International Journal of Computer Vision.

[13]  Roland Siegwart,et al.  Finding the Exact Rotation between Two Images Independently of the Translation , 2012, ECCV.

[14]  Martin Byröd,et al.  A Column-Pivoting Based Strategy for Monomial Ordering in Numerical Gröbner Basis Calculations , 2008, ECCV.

[15]  Joe Kileel,et al.  Minimal Problems for the Calibrated Trifocal Variety , 2016, SIAM J. Appl. Algebra Geom..

[16]  Benjamin B. Kimia,et al.  Camera Pose Estimation Using First-Order Curve Differential Geometry , 2012, ECCV.

[17]  Levente Hajder,et al.  A Minimal Solution for Two-View Focal-Length Estimation Using Two Affine Correspondences , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Andrew Zisserman,et al.  Minimal projective reconstruction for combinations of points and lines in three views , 2004, Image Vis. Comput..

[19]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[20]  Timothy Duff,et al.  Solving polynomial systems via homotopy continuation and monodromy , 2016, ArXiv.

[21]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Torsten Sattler,et al.  Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Timothy Duff,et al.  Trifocal Relative Pose from Lines at Points and its Efficient Solution , 2019, ArXiv.

[24]  Long Quan,et al.  Minimal Projective Reconstruction Including Missing Data , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Timothy Duff,et al.  PLMP - Point-Line Minimal Problems in Complete Multi-View Visibility , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[27]  Yubin Kuang,et al.  Pose Estimation with Unknown Focal Length Using Points, Directions and Lines , 2013, 2013 IEEE International Conference on Computer Vision.

[28]  Peter F. Sturm,et al.  Pose estimation using both points and lines for geo-localization , 2011, 2011 IEEE International Conference on Robotics and Automation.

[29]  Pedro Miraldo,et al.  A Minimal Closed-Form Solution for Multi-Perspective Pose Estimation using Points and Lines , 2018, ECCV.

[30]  Pascal Monasse,et al.  Robust and Accurate Line- and/or Point-Based Pose Estimation without Manhattan Assumptions , 2016, ECCV.

[31]  Jonathan D. Hauenstein,et al.  Multiprojective witness sets and a trace test , 2015, Advances in Geometry.

[32]  Frank Sottile,et al.  Trace Test , 2016, 1608.00540.

[33]  Stergios I. Roumeliotis,et al.  Optimal estimation of vanishing points in a Manhattan world , 2011, 2011 International Conference on Computer Vision.

[34]  S. Shankar Sastry,et al.  Rank Conditions on the Multiple-View Matrix , 2004, International Journal of Computer Vision.

[35]  Frederik Schaffalitzky,et al.  Four Points in Two or Three Calibrated Views: Theory and Practice , 2006, International Journal of Computer Vision.

[36]  Benjamin B. Kimia,et al.  Multiview Differential Geometry of Curves , 2016, International Journal of Computer Vision.

[37]  Ahmed M. Elgammal,et al.  Line-based relative pose estimation , 2011, CVPR 2011.

[38]  Tomás Pajdla,et al.  Neighbourhood Consensus Networks , 2018, NeurIPS.

[39]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[40]  Vincent Lepetit,et al.  An Efficient Minimal Solution for Multi-camera Motion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  Bernd Sturmfels,et al.  Rigid multiview varieties , 2015, Int. J. Algebra Comput..

[42]  Zuzana Kukelova,et al.  Automatic Generator of Minimal Problem Solvers , 2008, ECCV.

[43]  Julie Delon,et al.  Accurate Junction Detection and Characterization in Natural Images , 2013, International Journal of Computer Vision.

[44]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[45]  Zuzana Kukelova,et al.  Beyond Grobner Bases: Basis Selection for Minimal Solvers , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[46]  Viktor Larsson,et al.  Polynomial Solvers for Saturated Ideals , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[47]  Torsten Sattler,et al.  InLoc: Indoor Visual Localization with Dense Matching and View Synthesis , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[48]  Michel Dhome,et al.  Determination of the Attitude of 3D Objects from a Single Perspective View , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Helder Araújo,et al.  Direct Solution to the Minimal Generalized Pose , 2015, IEEE Transactions on Cybernetics.

[50]  Peter F. Sturm,et al.  Minimal solutions for generic imaging models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[52]  Levente Hajder,et al.  Efficient Recovery of Essential Matrix From Two Affine Correspondences , 2018, IEEE Transactions on Image Processing.

[53]  Daniel Barath,et al.  Five-Point Fundamental Matrix Estimation for Uncalibrated Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[54]  Hongdong Li,et al.  An Efficient Hidden Variable Approach to Minimal-Case Camera Motion Estimation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Yubin Kuang,et al.  Stratified sensor network self-calibration from TDOA measurements , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[56]  Rekha R. Thomas,et al.  On the Existence of Epipolar Matrices , 2015, International Journal of Computer Vision.

[57]  Viktor Larsson,et al.  Efficient Solvers for Minimal Problems by Syzygy-Based Reduction , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Rekha R. Thomas,et al.  A Hilbert Scheme in Computer Vision , 2011, Canadian Journal of Mathematics.

[59]  Luke Oeding,et al.  The ideal of the trifocal variety , 2012, Math. Comput..

[60]  Homer H. Chen.  Pose Determination from Line-to-Plane Correspondences: Existence Condition and Closed-Form Solutions , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  Marc Pollefeys,et al.  A minimal solution to the rolling shutter pose estimation problem , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[62]  Jan-Michael Frahm,et al.  USAC: A Universal Framework for Random Sample Consensus , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Zuzana Kukelova,et al.  A Clever Elimination Strategy for Efficient Minimal Solvers , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  M. Oskarsson,et al.  Classifying and solving minimal structure and motion problems with missing data , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[65]  Brett Browning,et al.  Evaluating Pose Estimation Methods for Stereo Visual Odometry on Robots , 2010 .