Panoramic EPI generation and analysis of video from a moving platform with vibration

This paper presents a novel approach for generating and analyzing epipolar plane images (EPIs) from video sequences taken from a moving platform subject to vibration so that the 3D model of an arbitrary scene can be constructed. Two problems are solved in our approach: (1) how to generate EPIs from video under a more general motion than a pure translation; (2) how to analyze the huge amount of data in the EPIs robustly and efficiently. For the first problem, a 3D image stabilization method is proposed which decouples the vibration from the vehicle's motion so that good EPIs and panoramic view images (PVIs) can be generated. For the second problem, we propose an efficient panoramic EPI analysis (PEPIA) method in which only one scanline of each EPI is processed. The PEPIA combines advantages of PVIs and EPIs and consists of three important steps: locus orientation detection, motion boundary localization, and occlusion/resolution recovery. The output of the PEPIA-a layered 3D panorama, is very useful in visual navigation and virtual reality modeling. Since camera calibration, image segmentation, feature extraction and matching are avoided, all the proposed algorithms are fully automatic and rather general. Results on real image sequences are given.

[1]  Kristin J. Dana,et al.  Real-time scene stabilization and mosaic construction , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[2]  Harpreet S. Sawhney,et al.  VideoBrush/sup TM/: experiences with consumer video mosaicing , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[3]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.

[4]  David W. Murray.  Recovering Range Using Virtual Multicamera Stereo , 1995, Comput. Vis. Image Underst..

[5]  Zhigang Zhu.  CAMERA STABILIZATION BASED ON 2.5D MOTION ESTIMATION AND INERTIAL MOTION FILTERING , 1998 .

[6]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[7]  Heung-Yeung Shum,et al.  Panoramic Image Mosaics , 1998 .

[8]  Xueyin Lin,et al.  Constructing 3D natural scene from video sequences with vibrated motions , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[9]  Seth Teller,et al.  Automatic Extraction of Textured Vertical Facades from Pose Imagery , 1998 .

[10]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[11]  Saied Moezzi,et al.  Virtual View Generation for 3D Digital Video , 1997, IEEE Multim..

[12]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[14]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[15]  Rama Chellappa,et al.  Fast electronic digital image stabilization , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[16]  Zhigang Zhu.  Environment modeling for visual navigation , 1997 .

[17]  Michitaka Hirose.  Image-Based Virtual World Generation , 1997, IEEE Multim..

[18]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Mei Han,et al.  Interactive construction of 3D models from panoramic mosaics , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).