Scene modelling from sparse 3D data

Sparse 3D measurements of real scenes are readily estimated from N-view image sequences using structure-from-motion techniques. In this paper, we present a geometric theory for reconstruction of surface models from sparse 3D data captured from N camera views. Based on this theory, we introduce a general N-view algorithm for reconstruction of 3D models of arbitrary scenes from sparse data. This algorithm reconstructs a surface model which converges to an approximation of the real scene surfaces and is consistent with the feature visibility in all N-views. To achieve efficient reconstruction independent of the number of views a recursive reconstruction algorithm is developed which integrates the feature visibility independently for each view. This approach is shown to converge to an approximation of the real scene structure and have a computational cost which is linear in the number of views. It is assumed that structure-from-motion estimates of 3D feature locations are consistent with the multiple view visual geometry and do not contain outliers. Uncertainty in 3D feature estimates is incorporated in the feature visibility to achieve reliable reconstruction in the presence of noise inherent in estimates of 3D scene structure from real image sequences. Results are presented for reconstruction of both real and synthetic scenes together with an evaluation of the reconstruction performance in the presence of noise. The algorithm presented in this paper provides a reliable and computationally efficient approach to model reconstruction from sparse 3D scene data.