View-synthesis from uncalibrated cameras and parallel planes

Abstract In this paper we present a view synthesis algorithm for the generation of novel intermediate views that smoothly interpolate between two reference images of the same scene acquired from two different point of views. If the acquisition system is not calibrated, there is no way to access the Euclidean frame and therefore it is impossible to determine the position and the orientation of the cameras as well as the three dimensional reconstruction of the imaged scene. Under the assumption of equal internal calibration of the two cameras, it is proved that the knowledge of the infinite homography is a sufficient condition for generating physically valid synthetic images from a pair of uncalibrated images. The view synthesis problem is therefore brought to the simpler well-known task of the estimation of the infinite homography through a set of geometric constraints existing in the scene. The work is complemented by the introduction of two novel techniques for estimating the infinite homography. This contributes to provide a variety of possibilities for such estimation, depending on the geometric constraints that can be identified in the imaged scene. The validity of the work is supported by a set of simulations on a synthetic three-dimensional scene and experiments on real data as well.