Accurate and efficient ground-to-aerial model alignment

Abstract To produce a complete 3D reconstruction of a large-scale architectural scene, both ground and aerial images are usually captured. A common approach is to first reconstruct the models from different image sources separately, and align them afterwards. Using this pipeline, this work proposes an accurate and efficient approach for ground-to-aerial model alignment in a coarse-to-fine manner. First, both the ground model and aerial model are transformed into the geo-referenced coordinate system using GPS meta-information for coarse alignment. Then, the coarsely aligned models are refined by a similarity transformation that is estimated based on 3D point correspondences between them, and the 3D point correspondences are determined in a 2D-image-matching manner by considering the rich textural and contextual information in the 2D images. Due to the dramatic differences in viewpoint and scale between ground and aerial images, which make matching them directly nearly impossible, we perform an intermediate view-synthesis step to mitigate the matching difficulty. To this end, the following three key issues are addressed: (a) selecting a suitable subset of aerial images to cover the ground model properly; (b) synthesizing images from the ground model under the viewpoints of the selected aerial images; and finally, (c) obtaining the 2D point matches between the synthesized images and the selected aerial images. The experimental results show that the proposed model alignment approach is quite effective and outperforms several state-of-the-art techniques in terms of both accuracy and efficiency.