Bundle Adjustment - A Modern Synthesis

This paper is a survey of the theory and methods of photogrammetric bundle adjustment, aimed at potential implementors in the computer vision community. Bundle adjustment is the problem of refining a visual reconstruction to produce jointly optimal structure and viewing parameter estimates. Topics covered include: the choice of cost function and robustness; numerical optimization including sparse Newton methods, linearly convergent approximations, updating and recursive methods; gauge (datum) invariance; and quality control. The theory is developed for general robust cost functions rather than restricting attention to traditional nonlinear least squares.

[1]  K. F. Gauss,et al.  Theoria combinationis observationum erroribus minimis obnoxiae , 1823 .

[2]  F. R. Helmert.  Die Mathematischen und physikalischen Theorieen der höheren Geodäsie , 1880 .

[3]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[4]  M. Kendall.  Theoretical Statistics , 1956, Nature.

[5]  J. Davenport.  Editor , 1960 .

[6]  W. Baarda,et al.  Statistical concepts in geodesy. , 1967 .

[7]  W. Baarda.  A testing procedure for use in geodetic networks. , 1968 .

[8]  Adrien-Marie Legendre,et al.  Nouvelles méthodes pour la détermination des orbites des comètes , 1970 .

[9]  Ian P. King,et al.  An automatic reordering scheme for simultaneous equations derived from network systems , 1970 .

[10]  G. R. Walsh,et al.  Methods Of Optimization , 1976 .

[11]  Duane C. Brown,et al.  Close-Range Camera Calibration , 1971 .

[12]  A. George.  Nested Dissection of a Regular Finite Element Mesh , 1973 .

[13]  Joseph Wai-Hung Liu,et al.  On reducing the profile of sparse symmetric matrices. , 1976 .

[14]  Manuscripta geodaetica , 1976 .

[15]  G. Golub,et al.  Large scale geodetic least squares adjustment by dissection and orthogonal decomposition , 1979 .

[16]  Y. Saad.  On the Rates of Convergence of the Lanczos and the Block-Lanczos Methods , 1980 .

[17]  Soren W. Henriksen,et al.  Manual of photogrammetry , 1980 .

[18]  Alan George,et al.  Computer Solution of Large Sparse Positive Definite , 1981 .

[19]  Philip E. Gill,et al.  Practical optimization , 1981 .

[20]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[21]  J. Blais.  Linear Least-Squares Computations Using Givens Transformations , 1983 .

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  A. George,et al.  A Comparison of Some Methods for Solving Sparse Linear Least-Squares Problems , 1983 .

[24]  A. Gruen.  Algorithmic aspects in on-line triangulation , 1985 .

[25]  Armin Gruen,et al.  Adaptive Least Squares Correlation With Geometrical Constraints , 1986, Other Conferences.

[26]  W. Föstner.  Reliability analysis of parameter estimation in linear models with application to mensuration problems in computer vision , 1987 .

[27]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[28]  A. George,et al.  Householder reflections versus givens rotations in sparse orthogonal decomposition , 1987 .

[29]  Stephen J. Wright,et al.  Optimization Software Guide , 1987 .

[30]  P. Boggs,et al.  A Stable and Efficient Algorithm for Nonlinear Orthogonal Distance Regression , 1987 .

[31]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[32]  C. Ghilani,et al.  Adjustment Computations: Statistics and Least Squares in Surveying and GIS , 1987 .

[33]  G. Stewart,et al.  Theory of the Combination of Observations Least Subject to Errors , 1987 .

[34]  Wolfgang Förstner.  Reliability analysis of parameter estimation in linear models with applications to mensuration problems in computer vision , 1987, Comput. Vis. Graph. Image Process..

[35]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[36]  H. M. Karara,et al.  Non-topographic photogrammetry , 1989 .

[37]  A. Greenbaum.  Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences , 1989 .

[38]  K. Holm.  Test of algorithms for sequential adjustment in online phototriangulation , 1989 .

[39]  Mengxiang Li.  Hierarchical multi-point matching with simultaneous detection and location of breaklines , 1990 .

[40]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Emmanuel P. Baltsavias,et al.  Multiphoto geometrically constrained matching , 1991 .

[42]  Janet E. Rogers,et al.  User's reference guide for ODRPACK version 2.01:: software for weighted orthogonal distance regression , 1992 .

[43]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[44]  P. D. Jonge.  A comparative study of algorithms for reducing the fill-in during Cholesky factorization , 1992 .

[45]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[47]  O. Faugeras,et al.  On determining the fundamental matrix : analysis of different methods and experimental results , 1993 .

[48]  Scott Mason.  Expert system based design of photogrammetric networks , 1994 .

[49]  A. Dermanis.  The photogrammetric inner constraints , 1994 .

[50]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[51]  Rajiv Gupta,et al.  Linear Pushbroom Cameras , 1994, ECCV.

[52]  David W. Murray,et al.  A unifying framework for structure and motion recovery from image sequences , 1995, Proceedings of IEEE International Conference on Computer Vision.

[53]  Scott Joseph Mason.  Expert system-based design of close-range photogrammetric networks , 1995 .

[54]  Harry Shum,et al.  Motion estimation with quadtree splines , 1995, Proceedings of IEEE International Conference on Computer Vision.

[55]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[56]  Richard Hartley.  An object-oriented approach to scene reconstruction , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).

[57]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[58]  David W. Murray,et al.  Active Camera Calibration for a Head-Eye Platform Using the Variable State-Dimension Filter , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Richard Szeliski,et al.  Shape Ambiguities in Structure From Motion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  B. Triggs.  A New Approach to Geometric Fitting , 1996 .

[61]  K. Atkinson.  Close Range Photogrammetry and Machine Vision , 1996 .

[62]  Josef Jansa,et al.  Advanced methods and applications , 1997 .

[63]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[64]  Harry Shum,et al.  A Parallel Feature Tracker for Extended Image Sequences , 1997, Comput. Vis. Image Underst..

[65]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[66]  George Karypis,et al.  Multilevel k-way Partitioning Scheme for Irregular Graphs , 1998, J. Parallel Distributed Comput..

[67]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[68]  Joseph W. H. Liu,et al.  Robust Ordering of Sparse Matrices using Multisection , 1998 .

[69]  Naoya Ohta,et al.  Optimal Robot Self-Localization and Reliability Evaluation , 1998, ECCV.

[70]  Bruce Hendrickson,et al.  Improving the Run Time and Quality of Nested Dissection Ordering , 1998, SIAM J. Sci. Comput..

[71]  Bill Triggs.  Optimal Estimation of Matching Constraints , 1998, SMILE.

[72]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[73]  Takeo Kanade,et al.  A unified factorization algorithm for points, line segments and planes with uncertainty models , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[74]  Philip F. McLauchlan,et al.  Gauge Independence in Optimization Algorithms for 3D Vision , 1999, Workshop on Vision Algorithms.

[75]  David A. Forsyth,et al.  Bayesian structure from motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[76]  James Demmel,et al.  LAPACK Users' Guide, Third Edition , 1999, Software, Environments and Tools.

[77]  Qian Chen,et al.  Efficient iterative solution to M-view projective reconstruction problem , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[78]  P. Anandan,et al.  Direct Recovery of Planar-Parallax from Multiple Frames , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  X. Wang,et al.  SEPARATE ADJUSTMENT OF CLOSE RANGE PHOTOGRAMMETRIC MEASUREMENTS , 1999 .

[80]  P. McLauchlan.  Gauge invariance in projective 3D reconstruction , 1999, Proceedings IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (MVIEW'99).

[81]  Takeo Kanade,et al.  Uncertainty Modeling for Optimal Structure from Motion , 1999, Workshop on Vision Algorithms.

[82]  Philip F. McLauchlan,et al.  A batch/recursive algorithm for 3D scene reconstruction , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[83]  Karl Kraus,et al.  Fundamentals and standard processes , 2000 .

[84]  R. Hartley,et al.  The Cubic Rational Polynomial Camera Model , 2001 .

[85]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[86]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..