Dynamic 3D Urban Scene Modeling Using Multiple Pushbroom Mosaics

In this paper, a unified, segmentation-based approach is proposed to deal with both stereo reconstruction and moving objects detection problems using multiple stereo mosaics. Each set of parallel-perspective (pushbroom) stereo mosaics is generated from a video sequence captured by a single video camera. First a color-segmentation approach is used to extract the so-called natural matching primitives from a reference view of a pair of stereo mosaics to facilitate both 3D reconstruction of textureless urban scenes and man-made moving targets (e.g. vehicles). Multiple pairs of stereo mosaics are used to improve the accuracy and robustness in 3D recovery and occlusion handling. Moving targets are detected by inspecting their 3D anomalies, either violating the epipolar geometry of the pushbroom stereo or exhibiting abnormal 3D structure. Experimental results on both simulated and real video sequences are provided to show the effectiveness of our approach.

[1]  Rajiv Gupta,et al.  Linear Pushbroom Cameras , 1994, ECCV.

[2]  Harry Shum,et al.  Stereo reconstruction from multiperspective panoramas , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[5]  Allen R. Hanson,et al.  Generalized parallel-perspective stereo mosaics from airborne video , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Emanuele Trucco,et al.  Efficient stereo with multiple windowing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Yael Pritch,et al.  Omnistereo: Panoramic Stereo Imaging , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ramakant Nevatia,et al.  Interactive 3D building modeling using a hierarchical representation , 2003, First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 2003. HLK 2003..

[9]  Shmuel Peleg,et al.  Universal mosaicing using pipe projection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[10]  Yi Deng,et al.  A symmetric patch-based correspondence model for occlusion handling , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Hai Tao,et al.  A background layer model for object tracking through occlusion , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  Saburo Tsuji,et al.  Panoramic representation for route recognition by a mobile robot , 1992, International Journal of Computer Vision.

[14]  Mongi A. Abidi,et al.  SAFER under vehicle inspection through video mosaic building , 2004, Ind. Robot.

[15]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Mubarak Shah,et al.  Motion layer extraction in the presence of occlusion using graph cuts , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Allen R. Hanson,et al.  Parallel-perspective stereo mosaics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[18]  Li Zhao,et al.  3D Measurements in Cargo Inspection with a Gamma-Ray Linear Pushbroom Stereo System , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[19]  Allen R. Hanson,et al.  Mosaic generation for under vehicle inspection , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[20]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  Allen R. Hanson,et al.  LAMP: 3D layered, adaptive-resolution, and multi-perspective panorama - a new scene representation , 2004, Comput. Vis. Image Underst..

[22]  Takeo Kanade,et al.  A subspace approach to layer extraction , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  Mathias Kölsch,et al.  Emerging Topics in Computer Vision , 2004 .

[24]  Harry Shum,et al.  Parallel projections for stereo reconstruction , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  Robert T. Collins,et al.  Mean-shift blob tracking through scale space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..