Universal mosaicing using pipe projection

Video mosaicing is commonly used to increase the visual field by pasting together many video frames. Existing mosaicing methods are effective only in very limited cases where the image motion is almost a uniform translation or the camera performs a pure pan. Forward camera motion or camera zoom are very problematic for traditional mosaicing. A mosaicing methodology to allow image mosaicing in the most general cases is presented, where frames in the video sequence are transformed such that the optical flow becomes parallel. This transformation is an oblique projection of the image into a "viewing pipe" whose central axis is the trajectory of the camera. The "pipe projection" enables to define high quality mosaicing even for the most challenging cases of forward motion and of zoom. In addition view interpolation, generating dense intermediate views is used to overcome parallax effects.

[1]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[2]  Annick Montanvert,et al.  Image mosaicking applied to three-dimensional surfaces , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[3]  Lance Williams,et al.  View Interpolation for Image Synthesis , 1993, SIGGRAPH.

[4]  Steven M. Seitz,et al.  Physically-valid view synthesis by image interpolation , 1995, Proceedings IEEE Workshop on Representation of Visual Scenes (In Conjunction with ICCV'95).

[5]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[6]  Steve Mann,et al.  Virtual bellows: constructing high quality stills from video , 1994, Proceedings of 1st International Conference on Image Processing.

[7]  Kristin J. Dana,et al.  Real-time scene stabilization and mosaic construction , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[8]  Rajiv Gupta,et al.  Linear Pushbroom Cameras , 1994, ECCV.

[9]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[10]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[11]  Harpreet S. Sawhney,et al.  Model-based 2D&3D dominant motion estimation for mosaicing and video representation , 1995, Proceedings of IEEE International Conference on Computer Vision.

[12]  Michal Irani,et al.  Detecting and Tracking Multiple Moving Objects Using Temporal Integration , 1992, ECCV.

[13]  Amnon Shashua,et al.  Robust recovery of camera rotation from three frames , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.