Fast shape-from-template using local features

Reconstructing the 3D surface of an object using only a single image is a challenging task, which has recently attracted attention. In this paper, a template-based approach is presented to reconstruct the surface of an isometric deformable object. The proposed approach brings a solution for a class of computer vision problems named shape-from-template (SfT). In SfT, the goal is to solve single-image reconstruction for an object given its 3D template model in some rest shape. To this end, corresponding keypoints between the template and the so-called deformed image are first established. Then, a very fast method is used to estimate the first-order differential flow around the extracted keypoint pairs as an affine transformation. This is done using the keypoint pairs’ surrounding texture patch. In our method, we estimate this affine transformation using the keypoint pairs’ closest neighbors. This is both faster and more stable. Finally, the depth of each keypoint in the deformed image is estimated from its associated affine transformation. The robustness of keypoint matching is essential to the process. Indeed, outliers defeat depth estimation dramatically. We propose two new approaches to detect and remove the possible outliers based on geometrical properties of the matched keypoints. These two geometrical outlier removal approaches are faster than existing ones and can be used with almost any image descriptor. Experimental results show that the proposed approaches are very effective and outperform existing ones.