Structure-from-Motion Revisited

Incremental Structure-from-Motion is a prevalent strategy for 3D reconstruction from unordered image collections. While incremental reconstruction systems have tremendously advanced in all regards, robustness, accuracy, completeness, and scalability remain the key problems towards building a truly general-purpose pipeline. We propose a new SfM technique that improves upon the state of the art to make a further step towards this ultimate goal. The full reconstruction pipeline is released to the public as an open-source implementation.

[1]  C. A. HART,et al.  Manual of Photogrammetry , 1947, Nature.

[2]  Soren W. Henriksen,et al.  Manual of photogrammetry , 1980 .

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1995, Int. J. Robotics Res..

[5]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[6]  Philip H. S. Torr.  An assessment of information criteria for motion model selection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Ian D. Reid,et al.  Self-Calibration of a Rotating Camera with Varying Intrinsic Parameters , 1998, BMVC.

[8]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[9]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[10]  Frank Dellaert,et al.  Structure from motion without correspondence , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[11]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[12]  Andrew Zisserman,et al.  Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps?" , 2002, ECCV.

[13]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[15]  Richard I. Hartley,et al.  L-8Minimization in Geometric Reconstruction Problems , 2004, CVPR.

[16]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[17]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[18]  Christian Beder,et al.  Determining an Initial Image Pair for Fixing the Scale of a 3D Reconstruction from an Image Sequence , 2006, DAGM-Symposium.

[19]  Jan-Michael Frahm,et al.  RANSAC for (Quasi-)Degenerate data (QDEGSAC) , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[21]  Frank Dellaert,et al.  Out-of-Core Bundle Adjustment for Large-Scale 3D Reconstruction , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Hongdong Li,et al.  A practical algorithm for L triangulation with outliers , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[24]  Richard I. Hartley,et al.  A Fast Optimal Algorithm for L 2 Triangulation , 2007, ACCV.

[25]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[26]  Steven M. Seitz,et al.  Fast algorithms for L∞ problems in multiview geometry , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Zuzana Kukelova,et al.  A general solution to the P4P problem for camera with unknown focal length , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  YANQING CHEN,et al.  Algorithm 8 xx : CHOLMOD , supernodal sparse Cholesky factorization and update / downdate ∗ , 2006 .

[29]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Shengyong Chen,et al.  Active Sensor Planning for Multiview Vision Tasks , 2008 .

[31]  Jan-Michael Frahm,et al.  Next Best View Planning for Active Model Improvement , 2009, BMVC.

[32]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[33]  H. Bischof,et al.  From structure-from-motion point clouds to fast location recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[35]  S. Lazebnik,et al.  Building Rome on a Cloudless Day ( ECCV 2010 ) , 2010 .

[36]  Andrea Fusiello,et al.  Improving the efficiency of hierarchical structure-and-motion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[38]  Richard Szeliski,et al.  Bundle Adjustment in the Large , 2010, ECCV.

[39]  Anders P. Eriksson,et al.  Outlier removal using duality , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Daniel P. Huttenlocher,et al.  Location Recognition Using Prioritized Feature Matching , 2010, ECCV.

[41]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[42]  Gang Hua,et al.  Discriminative Learning of Local Image Descriptors , 1990, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR 2011.

[44]  Noah Snavely,et al.  Scene Reconstruction and Visualization from Internet Photo Collections: A Survey , 2011, IPSJ Trans. Comput. Vis. Appl..

[45]  Sameer Agarwal,et al.  Visibility Based Preconditioning for bundle adjustment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Anders Heyden,et al.  Covariance Propagation and Next Best View Planning for 3D Reconstruction , 2012, ECCV.

[47]  Rekha R. Thomas,et al.  A QCQP Approach to Triangulation , 2012, ECCV.

[48]  Jan-Michael Frahm,et al.  Comparative Evaluation of Binary Features , 2012, ECCV.

[49]  Johannes Gehrke,et al.  MatchMiner: Efficient Spanning Structure Mining in Large Image Collections , 2012, ECCV.

[50]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[51]  Frank Dellaert,et al.  Mining Structure Fragments for Smart Bundle Adjustment , 2014, BMVC.

[52]  Noah Snavely,et al.  Robust Global Translations with 1DSfM , 2014, ECCV.

[53]  Konrad Schindler,et al.  VocMatch: Efficient Multiview Correspondence for Structure from Motion , 2014, ECCV.

[54]  Yee-Hong Yang,et al.  Robust multi-view L2 triangulation via optimal inlier selection and 3D structure refinement , 2014, Pattern Recognit..

[55]  Bastian Leibe,et al.  Fixing WTFs: Detecting Image Matches Caused by Watermarks, Timestamps, and Frames in Internet Photos , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[56]  Jan-Michael Frahm,et al.  PAIGE: PAirwise Image Geometry Encoding for improved efficiency in Structure-from-Motion , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Jan-Michael Frahm,et al.  Efficient Two-View Geometry Classification , 2015, GCPR.

[58]  Changchang Wu,et al.  Structure from Motion Using Structure-Less Resection , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[59]  Tobias Höllerer,et al.  Optimizing the Viewing Graph for Structure-from-Motion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[60]  Jan-Michael Frahm,et al.  From single image query to detailed 3D reconstruction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Jan-Michael Frahm,et al.  Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset) , 2015, CVPR 2015.

[62]  Jan-Michael Frahm,et al.  From Dusk Till Dawn: Modeling in the Dark , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).