New View Generation with a Bi-centric Camera

We propose a novel method for new view generation from a rectified sequence of images. Our new images correspond to a new camera model, which we call a bi-centric camera; in this model the centers of horizontal and vertical projections lie in different locations on the camera's optical axis. This model reduces to the regular pinhole camera when the two projection centers coincide, and the pushbroom camera when one projection center lies at infinity. We first analyze the properties of this camera model. We then show how to generate new bi-centric views from vertical cuts in the epipolar volume of a rectified sequence. Every vertical cut generates a new bi-centric view, where the specific parameters of the cut determine the location of the projection centers. We discuss and demonstrate applications, including the generation of images where the virtual camera lies behind occluding surfaces (e.g., behind the back wall of a room), and in unreachable positions (e.g., in front of a glass window). Our final application is the generation of movies taken by a simulated forward moving camera, using as input a movie taken by a sideways moving camera.

[1]  O. Faugeras.  Three-Dimensional Computer Vision , 1993 .

[2]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[3]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[4]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[5]  Amnon Shashua,et al.  Novel view synthesis in tensor space , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Ian D. Reid,et al.  Duality, Rigidity and Planar Parallax , 1998, ECCV.

[8]  Daphna Weinshall,et al.  From Reference Frames to Reference Planes: Multi-View Parallax Geometry and Applications , 1998, ECCV.

[9]  Sing Bing Kang,et al.  Survey of image-based rendering techniques , 1998, Electronic Imaging.

[10]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.

[11]  Katsushi Ikeuchi,et al.  Arbitrary view position and direction rendering for large-scale scenes , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[12]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[13]  Tomás Pajdla.  Stereo with Oblique Cameras , 2004, International Journal of Computer Vision.

[14]  Steven M. Seitz,et al.  The Space of All Stereo Images , 2004, International Journal of Computer Vision.