Practical Projective Structure from Motion (P2SfM)

This paper presents a solution to the Projective Structure from Motion (PSfM) problem able to deal efficiently with missing data, outliers and, for the first time, large scale 3D reconstruction scenarios. By embedding the projective depths into the projective parameters of the points and views, we decrease the number of unknowns to estimate and improve computational speed by optimizing standard linear Least Squares systems instead of homogeneous ones. In order to do so, we show that an extension of the linear constraints from the Generalized Projective Reconstruction Theorem can be transferred to the projective parameters, ensuring also a valid projective reconstruction in the process. We use an incremental approach that, starting from a solvable sub-problem, incrementally adds views and points until completion with a robust, outliers free, procedure. Experiments with simulated data shows that our approach is performing well, both in term of the quality of the reconstruction and the capacity to handle missing data and outliers with a reduced computational time. Finally, results on real datasets shows the ability of the method to be used in medium and large scale 3D reconstruction scenarios with high ratios of missing data (up to 98%).

[1]  Andrea Fusiello,et al.  Hierarchical structure-and-motion recovery from uncalibrated images , 2015, Comput. Vis. Image Underst..

[2]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Stéphane Christy,et al.  Euclidean Shape and Motion from Multiple Perspective Views by Affine Iterations , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[5]  Bill Triggs,et al.  Factorization methods for projective structure and motion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Magnus Oskarsson,et al.  Trust No One: Low Rank Matrix Factorization Using Hierarchical RANSAC , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Hongdong Li,et al.  Element-Wise Factorization for N-View Projective Reconstruction , 2010, ECCV.

[8]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[9]  David J. Kriegman,et al.  Autocalibration via Rank-Constrained Estimation of the Absolute Quadric , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Carl Olsson,et al.  Stable Structure from Motion for Unordered Image Collections , 2011, SCIA.

[11]  Tomás Pajdla,et al.  3D reconstruction by fitting low-rank matrices with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Nicolas Gillis,et al.  Low-Rank Matrix Approximation with Weights or Missing Data Is NP-Hard , 2010, SIAM J. Matrix Anal. Appl..

[13]  Martial Hebert,et al.  Provably-convergent iterative methods for projective structure from motion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Fredrik Kahl,et al.  Structure from Motion with Missing Data is NP-Hard , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[16]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[17]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[18]  Richard I. Hartley,et al.  Critical Configurations for Projective Reconstruction from Multiple Views , 2005, International Journal of Computer Vision.

[19]  Fumiaki Tomita,et al.  A Factorization Method for Projective and Euclidean Reconstruction from Multiple Perspective Views via Iterative Depth Estimation , 1998, ECCV.

[20]  Prateek Jain,et al.  Universal Matrix Completion , 2014, ICML.

[21]  Hongdong Li,et al.  Projective Multiview Structure and Motion from Element-Wise Factorization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Andrew W. Fitzgibbon,et al.  Damped Newton algorithms for matrix factorization with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Qian Chen,et al.  Efficient iterative solution to M-view projective reconstruction problem , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[24]  Stephen J. Wright,et al.  Online algorithms for factorization-based structure from motion , 2013, IEEE Winter Conference on Applications of Computer Vision.

[25]  Andrew W. Fitzgibbon,et al.  Projective Bundle Adjustment from Arbitrary Initialization Using the Variable Projection Method , 2016, ECCV.

[26]  Richard I. Hartley,et al.  Iterative Extensions of the Sturm/Triggs Algorithm: Convergence and Nonconvergence , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Magnus Oskarsson,et al.  On the minimal problems of low-rank matrix factorization , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Jochen Trumpf,et al.  A Generalized Projective Reconstruction Theorem and Depth Constraints for Projective Factorization , 2015, International Journal of Computer Vision.

[29]  Anders Heyden,et al.  An iterative factorization method for projective structure and motion from image sequences , 1999, Image Vis. Comput..

[30]  Aleix M. Martínez,et al.  Low-Rank Matrix Fitting Based on Subspace Perturbation Analysis with Applications to Structure from Motion , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.