3D Modelling of Static Environments Using Multiple Spherical Stereo

We propose a 3D modelling method from multiple pairs of spherical stereo images. A static environment is captured as a vertical stereo pair with a rotating line scan camera at multiple locations and depth fields are extracted for each pair using spherical stereo geometry. We propose a new PDE-based stereo matching method which handles occlusion and over-segmentation problem in highly textured regions. In order to avoid cumbersome camera calibration steps, we extract a 3D rigid transform using feature matching between views and fuse all models into one complete mesh. A reliable surface selection algorithm for overlapped surfaces is proposed for merging multiple meshes in order to keep surface details while removing outliers. The performances of the proposed algorithms are evaluated against ground-truth from LIDAR scans.

[1]  Jiří Matas,et al.  Computer Vision - ECCV 2004 , 2004, Lecture Notes in Computer Science.

[2]  Mariette Yvinec,et al.  Surface Reconstruction from Multi-View Stereo of Large-Scale Outdoor Scenes , 2010, Int. J. Virtual Real..

[3]  Michael J. Black,et al.  Learning Optical Flow , 2008, ECCV.

[4]  Naokazu Yokoya,et al.  3D modeling of outdoor environments by integrating omnidirectional range and color images , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[5]  Daphna Weinshall,et al.  Realtime IBR with omnidirectional crossed-slits projection , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[7]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[8]  Naokazu Yokoya,et al.  3D reconstruction of outdoor environments from omnidirectional range and color images , 2005, IS&T/SPIE Electronic Imaging.

[9]  David Puig,et al.  Acquisition, modelling and rendering of very large urban environments , 2004 .

[10]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[11]  Shigang Li,et al.  Spherical stereo for the construction of immersive VR environment , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[12]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[13]  Kwanghoon Sohn,et al.  Hierarchical depth estimation for image synthesis in mixed reality , 2003, IS&T/SPIE Electronic Imaging.

[14]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Tony Lindeberg,et al.  Scale-Space Theory in Computer Vision , 1993, Lecture Notes in Computer Science.

[16]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Hans-Peter Seidel,et al.  PDE-Based Anisotropic Disparity-Driven Stereo Vision , 2008, VMV.

[18]  Joachim Weickert,et al.  A Review of Nonlinear Diffusion Filtering , 1997, Scale-Space.

[19]  Janusz Konrad,et al.  Occlusion-Aware Optical Flow Estimation , 2008, IEEE Transactions on Image Processing.

[20]  Nir A. Sochen,et al.  Variational Stereo Vision with Sharp Discontinuities and Occlusion Handling , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  J BeslPaul,et al.  A Method for Registration of 3-D Shapes , 1992 .

[22]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[23]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[24]  Marc Levoy,et al.  A volumetric method for building complex models from range images , 1996, SIGGRAPH.

[25]  Kwanghoon Sohn,et al.  Cost Aggregation and Occlusion Handling With WLS in Stereo Matching , 2008, IEEE Transactions on Image Processing.

[26]  Luc Van Gool,et al.  3D Urban Scene Modeling Integrating Recognition and Reconstruction , 2008, International Journal of Computer Vision.

[27]  Shigang Li,et al.  Full-View Spherical Image Camera , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[28]  Rachid Deriche,et al.  Symmetrical Dense Optical Flow Estimation with Occlusions Detection , 2002, International Journal of Computer Vision.

[29]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[30]  Adrian Hilton,et al.  Environment modelling using spherical stereo imaging , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[31]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[32]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[33]  Reinhard Koch,et al.  Automated reconstruction of 3D scenes from sequences of images , 2000 .

[34]  Mariette Yvinec,et al.  Surface Reconstruction from Multi-View Stereo , 2009 .

[35]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[36]  Shree K. Nayar,et al.  Catadioptric omnidirectional camera , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[38]  Mads Nielsen,et al.  Computer Vision — ECCV 2002 , 2002, Lecture Notes in Computer Science.

[39]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Rachid Deriche,et al.  Dense Disparity Map Estimation Respecting Image Discontinuities: A PDE and Scale-Space BasedApproach , 2002, MVA.

[42]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.