Gauge invariance in projective 3D reconstruction

Bundle adjustment is a standard photogrammetric technique for optimizing the 3D reconstruction of a scene from multiple images. There is an inherent gauge (coordinate frame) ambiguity in 3D reconstruction that can seriously affect the convergence of bundle adjustment algorithms. We address this issue and show that a simple pre-conditioning step removes the effect of the choice of coordinate frame, and together with a set of enforced constraints on the reconstruction, achieves along with this invariance greatly increased convergence speed over existing methods. The new approach applies to all the well-known 3D reconstruction models: projective, affine and Euclidean. We develop the idea for projective reconstruction. The normalization stage partially removes the gauge freedom, reducing the coordinate frame choice from a general 3D homography to an orthogonal transformation; then constraints are incorporated in the bundle adjustment iterations that enforce the normalisation condition to first order. In the projective case the approach relies on a currently unproven matrix conjecture, which we nevertheless strongly believe to be correct, based on extensive experimental evidence.