Extension of Incomplete 3D for Arbitrary Multi-View-Synthesis

The proposed approach is motivated by applications which allow user navigation and individual viewpoint specification in shared virtual environments with telepresence quality. In this context, we present a synthesis method for arbitrary virtual views in a multi-view camera set-up. This method generates a close to realtime, view adaptable reconstruction of a 3dimensional, (3D), object taken from at least two cameras. In this method, we use the recently developed Incomplete 3D, (IC3D), technique, a disparity-based multiview representation for a weakly convergent camera setup in combination with the trilinear warping functions to describe new virtual camera positions. Previously, only virtual views inside the baseline could be described by IC3D. Hence, user movement in virtual environments (VE) was restricted. In order to create a virtual camera position and orientation outside the baseline we apply point correspondences across the two reference images to a trilinear tensor, built from the fundamental matrix between the reference views. The trilinearities provide a general warping function from the reference images to virtual view that is governed directly by the virtual camera parameters.