Mosaic-based 3D scene representation and rendering

In this paper we address the problem of fusing images from many video cameras or a moving video camera. The captured images have obvious motion parallax, but they will be aligned and integrated into a few mosaics with a large field-of-view (FOV) that preserve 3D information. We have developed a geometric representation that can re-organize the original perspective images into a set of parallel projections with different oblique viewing angles. In addition to providing a wide field of view, mosaics with various oblique views well represent occlusion regions that cannot be seen in a usual nadir view. Stereo pair(s) can be formed from a pair of mosaics with different oblique viewing angles and thus image-based 3D viewing can be achieved. This representation can be used as both an advanced video interface and a pre-processing step for 3D reconstruction.

[1]  Harry Shum,et al.  Parallel projections for stereo reconstruction , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[2]  Howard Schultz,et al.  Terrain reconstruction from widely separated images , 1995, Defense, Security, and Sensing.

[3]  Harry Shum,et al.  Stereo reconstruction from multiperspective panoramas , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[5]  Daphna Weinshall,et al.  Mosaicing New Views: The Crossed-Slits Projection , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Allen R. Hanson,et al.  Generalized parallel-perspective stereo mosaics from airborne video , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Allen R. Hanson,et al.  Generation and Error Characterization of Pararell-Perspective Stereo Mosaics from Real Video , 2003 .

[8]  Allen R. Hanson,et al.  Mosaic generation for under vehicle inspection , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[9]  Allen R. Hanson,et al.  An efficient method for geo-referenced video mosaicing for environmental monitoring , 2005, Machine Vision and Applications.

[10]  Yael Pritch,et al.  Omnistereo: Panoramic Stereo Imaging , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[12]  Shmuel Peleg,et al.  Mosaicing on Adaptive Manifolds , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Allen R. Hanson,et al.  LAMP: 3D layered, adaptive-resolution, and multi-perspective panorama - a new scene representation , 2004, Comput. Vis. Image Underst..

[14]  Hao Tang,et al.  3D and Moving Target Extraction from Dynamic Pushbroom Stereo Mosaics , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[15]  Richard Szeliski,et al.  Stereo reconstruction from multiperspective panoramas , 2004 .

[16]  Saburo Tsuji,et al.  Panoramic representation for route recognition by a mobile robot , 1992, International Journal of Computer Vision.