A Framework for Pencil-of-Points Structure-from-Motion

Our goal is to match contour lines between images and to recover structure and motion from those. The main difficulty is that pairs of lines from two images do not induce direct geometric constraint on camera motion. Previous work uses geometric attributes | orientation, length, etc. | for single or groups of lines. Our approach is based on using Pencil-of-Points (points on line) or pops for short. There are many advantages to using pops for structure-from-motion. The most important one is that, contrarily to pairs of lines, pairs of pops may constrain camera motion. We give a complete theoretical and practical framework for automatic structure-from-motion using pops | detection, matching, robust motion estimation, triangulation and bundle adjustment. For wide baseline matching, it has been shown that cross-correlation scores computed on neighbouring patches to the lines gives reliable results, given 2D homographic transformations to compensate for the pose of the patches. When cameras are known, this transformation has a 1-dimensional ambiguity. We show that when cameras are unknown, using pops lead to a 3-dimensional ambiguity, from which it is still possible to reliably compute cross-correlation. We propose linear and non-linear algorithms for estimating the fundamental matrix and for the multiple-view triangulation of pops. Experimental results are provided for simulated and real data.