Fast construction of dynamic and multi-resolution 360 degrees panoramas from video sequences

This paper presents a unified approach to automatically build dynamic and multi-resolution 360^o panoramic (DMP) representations from image sequences captured by hand-held cameras mainly undertaking rotation and zooming for natural scenes with moving targets. A simple (yet stable) rigid motion model and a closed-loop-based mosaicing algorithm are proposed to generate cylindrical mosaics automatically. Multi-resolution representations are built for interesting areas by separating zooming sub-sequences from a pan/zoom sequence. Moving objects are detected and separated from images based on motion information, and then more accurate contours are extracted using a modified active contour algorithm. The DMP construction method is fast, robust, and automatic, achieving five frames per second for image sequences with 384x288 color images on a Pentium III 800MHz PC. The construction of the DMP representation can be used in virtual reality, video surveillance, and very low bit-rate video coding.

[1]  Cordelia Schmid,et al.  Image matching with scale adjustment , 2004, Comput. Vis. Image Underst..

[2]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[3]  E. Riseman,et al.  Panoramic virtual stereo vision of cooperative mobile robots for localizing 3D moving objects , 2000, Proceedings IEEE Workshop on Omnidirectional Vision (Cat. No.PR00704).

[4]  Kenneth Turkowski,et al.  Creating image-based VR using a self-calibrating fisheye lens , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Shree K. Nayar,et al.  Generation of Perspective and Panoramic Video from Omnidirectional Video , 1997 .

[6]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[7]  Sing Bing Kang,et al.  Characterization of errors in compositing panoramic images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Allen R. Hanson,et al.  LAMP: 3D layered, adaptive-resolution, and multi-perspective panorama - a new scene representation , 2004, Comput. Vis. Image Underst..

[9]  Biswajit Bose,et al.  From video sequences to motion panoramas , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[10]  Allen R. Hanson,et al.  Generalized parallel-perspective stereo mosaics from airborne video , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shmuel Peleg,et al.  Fast panoramic stereo matching using cylindrical maximum surfaces , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  C. Schmid,et al.  Indexing based on scale invariant interest points , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[13]  Allen R. Hanson,et al.  An efficient method for geo-referenced video mosaicing for environmental monitoring , 2005, Machine Vision and Applications.

[14]  Xueyin Lin,et al.  Efficient Fourier-Based Approach for Detecting Orientations and Occlusions in Epipolar Plane Images for 3D Scene Modeling , 2005, International Journal of Computer Vision.

[15]  Shmuel Peleg,et al.  Panoramic mosaics by manifold projection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Allen R. Hanson,et al.  Fast generation of dynamic and multi-resolution 360/spl deg/ panorama from video sequences , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[17]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[19]  Roland T. Chin,et al.  Deformable Contours: Modeling and Extraction , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Zhigang Zhu.  CAMERA STABILIZATION BASED ON 2.5D MOTION ESTIMATION AND INERTIAL MOTION FILTERING , 1998 .

[21]  Mubarak Shah,et al.  A Fast algorithm for active contours and curvature estimation , 1992, CVGIP Image Underst..

[22]  Yael Pritch,et al.  Omnistereo: Panoramic Stereo Imaging , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Richard Szeliski,et al.  Stereo reconstruction from multiperspective panoramas , 2004 .

[24]  Ramesh C. Jain,et al.  Using Dynamic Programming for Solving Variational Problems in Vision , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Xueyin Lin,et al.  Panoramic EPI generation and analysis of video from a moving platform with vibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).