View synthesis and 3D reconstruction of piecewise planar scenes using intersection lines between the planes

A linear algorithm for synthesizing new views of piecewise planar objects without making an explicit 3D reconstruction is proposed, together with a simple algorithm for 3D reconstruction of the scene. It is shown how this can be done using only one image and information about the projection of the intersection lines between the object planes. These could either be estimated manually or more automatically using at least one more image. No calibration information is needed. A main idea in the paper is to work with textured planes. A patch in one image, corresponding to a planar surface in the scene, is transformed to a patch in another image by a homography. The generalized eigenvectors and eigenvalues of the homographies have geometrical interpretations that are used in the algorithms. To generate a new image, we calculate the homographies for each plane to the new image. The textures can then easily be mapped to the new image. The reconstruction is given as the equation of each plane in the corresponding homography to a certain image.

[1]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[2]  Daphna Weinshall,et al.  From Reference Frames to Reference Planes: Multi-View Parallax Geometry and Applications , 1998, ECCV.

[3]  Stefan Carlsson,et al.  Calibration, pose and novel views from single images of constrained scenes , 2000, Pattern Recognit. Lett..

[4]  Jitendra Malik,et al.  Reconstructing Polyhedral Models of Architectural Scenes from Photographs , 1996, ECCV.

[5]  David A. Forsyth,et al.  Extracting projective structure from single perspective views of 3D point sets , 1993, 1993 (4th) International Conference on Computer Vision.

[6]  J. G. Semple,et al.  Algebraic Projective Geometry , 1953 .

[7]  Steven M. Seitz,et al.  Physically-valid view synthesis by image interpolation , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[8]  Olivier D. Faugeras,et al.  What can two images tell us about a third one? , 1994, ECCV.

[9]  Amnon Shashua,et al.  Novel view synthesis in tensor space , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Ian D. Reid,et al.  Duality, Rigidity and Planar Parallax , 1998, ECCV.

[11]  Ken-ichi Anjyo,et al.  Tour into the picture: using a spidery mesh interface to make animation from a single image , 1997, SIGGRAPH.

[12]  Olivier D. Faugeras,et al.  3-D scene representation as a collection of images , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[13]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Ken-ichi Anjyo.  Tour into the picture , 1997, SIGGRAPH '97.