Bidirectional optimization for reconstructing 3D shape from an image sequence with missing data

Reconstruction of 3D structure from a 2D image sequence is very important for image understanding, image based rendering etc. especially, in practice, reconstruction algorithms that can handle occlusion are necessary. In this paper we extend the factorization algorithm and propose a method to reconstruct 3D shape from an image sequence with missing data. In our method, treating missing elements as unknown parameters, parametric representation of a measurement matrix is given. Since the rank of the ideal measurement matrix should be three, parameter values should be specified so that the discrepancy between the parametric matrix and its associated "closest" rank three matrix is minimized. We describe an iterative algorithm to determine the unknown parameter so as to give rise to the decreasing sequence of the discrepancy. The procedure is nothing but the iterative bidirectional projection between the model space and the observation space (parametric representation of incomplete data). Several experimental results are also provided.