Structure from Many Perspective Images with Occlusions ( Version 1 . 02 )

This paper proposes a method for recovery of projective shape and motion from multiple images by factorization of a matrix containing the images of all scene points. Compared to previous methods, this method can handle perspective views and occlusions at once. The projective depths of image points are estimated by the method of Sturm & Triggs [8] using epipolar geometry. Occlusions are solved by the extension of the method by Jacobs [6] for filling of missing data. This extension can exploit the geometry of perspective camera. Many ways of combining the two methods exist, and therefore several strategies of combining have been tested and the one with the best results is presented. The new method gives accurate results in practical situations, which is demonstrated with a series of experiments on laboratory and outdoor image sets. The theoretical contribution of this paper is the analysis of minimal recoverable configurations of missing points and projective depths.

[1]  Olivier D. Faugeras,et al.  What can be seen in three dimensions with an uncalibrated stereo rig , 1992, ECCV.

[2]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Peter F. Sturm,et al.  A Factorization Based Algorithm for Multi-Image Projective Structure and Motion , 1996, ECCV.

[4]  Anders Heyden.  Projective Structure and Motion from Image Sequences using Subspace Methods , 1997 .

[5]  David W. Jacobs,et al.  Linear fitting with missing data: applications to structure-from-motion and to characterizing intensity images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[7]  Martial Hebert,et al.  Iterative projective reconstruction from multiple views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.