Generating Mosaics with Minimum Distortions

Manifold mosaicing is a fast and robust way to summarize video sequences captured by a moving camera. It is also useful for rendering compelling 3D visualizations from a video without estimating the 3D structure of the scene. However, since the result mosaics are not perspective images, their geometry is inherently distorted. These mosaics are commonly referred to as multi-perspective images, or multi-perspective mosaics. In this paper we address the following question: Given a video captured by a moving camera, what is the best multi-perspective mosaic that can be generated from it? What is the mosaic with the best combination of large field-of-view and minimal geometric distortions? We define the necessary conditions for a good mosaic and a quantitive criterion for the geometric distortions, and derive analytically the optimal mosaic under this criterion. Results on video sequences confirm that indeed the optimal mosaic has significantly better quality than those generated by other techniques.

[1]  Hiroshi Ishiguro,et al.  Omni-Directional Stereo , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Saburo Tsuji,et al.  Panoramic representation for route recognition by a mobile robot , 1992, International Journal of Computer Vision.

[3]  David Salesin,et al.  Multiperspective panoramas for cel animation , 1997, SIGGRAPH.

[4]  Steven M. Seitz,et al.  The Space of All Stereo Images , 2004, International Journal of Computer Vision.

[5]  Marc Pollefeys,et al.  Multiple view geometry , 2005 .

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[7]  Harry Shum,et al.  Rendering with concentric mosaics , 1999, SIGGRAPH.

[8]  Shmuel Peleg,et al.  Mosaicing on Adaptive Manifolds , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Allen R. Hanson,et al.  Parallel-perspective stereo mosaics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[10]  Daphna Weinshall,et al.  Mosaicing New Views: The Crossed-Slits Projection , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Assaf Zomet,et al.  Least Distorted Mosaics , 2003 .

[12]  Yael Pritch,et al.  Omnistereo: Panoramic Stereo Imaging , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Shree K. Nayar,et al.  A perspective on distortions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Richard Szeliski,et al.  Stereo reconstruction from multiperspective panoramas , 2004 .

[15]  Tomás Pajdla.  Stereo with Oblique Cameras , 2004, International Journal of Computer Vision.

[16]  Rajiv Gupta,et al.  Linear Pushbroom Cameras , 1994, ECCV.

[17]  Paul Rademacher,et al.  Multiple-center-of-projection images , 1998, SIGGRAPH.

[18]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.