Manifold surface reconstruction of an environment from sparse Structure-from-Motion data

The majority of methods for the automatic surface reconstruction of an environment from an image sequence have two steps: Structure-from-Motion and dense stereo. From the computational standpoint, it would be interesting to avoid dense stereo and to generate a surface directly from the sparse cloud of 3D points and their visibility information provided by Structure-from-Motion. The previous attempts to solve this problem are currently very limited: the surface is non-manifold or has zero genus, the experiments are done on small scenes or objects using a few dozens of images. Our solution does not have these limitations. Furthermore, we experiment with hand-held or helmet-held catadioptric cameras moving in a city and generate 3D models such that the camera trajectory can be longer than one kilometer.

[1]  Eric Royer,et al.  Matching Planar Features for Robot Localization , 2009, ISVC.

[2]  Luc Van Gool,et al.  Fast Compact City Modeling for Navigation Pre-Visualization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Camillo J. Taylor.  Surface reconstruction from feature based stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Olivier D. Faugeras,et al.  Representing stereo data with the Delaunay triangulation , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[5]  Gabriel Taubin,et al.  Cutting and Stitching: Converting Sets of Polygons to Manifold Surfaces , 2001, IEEE Trans. Vis. Comput. Graph..

[6]  Olivier D. Faugeras,et al.  Complete Dense Stereovision Using Level Set Methods , 1998, ECCV.

[7]  Long Quan,et al.  A quasi-dense approach to surface reconstruction from uncalibrated images , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Andrew J. Davison,et al.  Live dense reconstruction with a single moving camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Maxime Lhuillier,et al.  Automatic scene structure and camera motion using a catadioptric system , 2008, Comput. Vis. Image Underst..

[11]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[12]  David Eppstein,et al.  Provably good mesh generation , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[13]  Maxime Lhuillier,et al.  Genus refinement of a manifold surface reconstructed by sculpting the 3d-Delaunay triangulation of Structure-from-Motion points , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[14]  Maxime Lhuillier,et al.  Surface Reconstruction of Scenes Using a Catadioptric Camera , 2011, MIRAGE.

[15]  Adrian Hilton,et al.  Scene modelling from sparse 3D data , 2005, Image Vis. Comput..

[16]  Pierre Alliez,et al.  Polygon Mesh Processing , 2010 .

[17]  Takeo Kanade,et al.  Image-consistent surface triangulation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Jean-Philippe Pons,et al.  Efficient Multi-View Reconstruction of Large-Scale Scenes using Interest Points, Delaunay Triangulation and Graph Cuts , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Tomás Pajdla,et al.  Multi-view reconstruction preserving weakly-supported surfaces , 2011, CVPR 2011.

[21]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[22]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[23]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[24]  Jean-Daniel Boissonnat,et al.  Geometric structures for three-dimensional shape representation , 1984, TOGS.

[25]  Gert Vegter,et al.  In handbook of discrete and computational geometry , 1997 .

[26]  Mathieu Desbrun,et al.  Removing excess topology from isosurfaces , 2004, TOGS.

[27]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[28]  Maxime Lhuillier,et al.  Incremental Reconstruction of Manifold Surface from Sparse Visual Mapping , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[29]  Francis Schmitt,et al.  Silhouette and stereo fusion for 3D object modeling , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[30]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[31]  Mark Meyer,et al.  Discrete Differential-Geometry Operators for Triangulated 2-Manifolds , 2002, VisMath.

[32]  Maxime Lhuillier.  ENVIRONMENT MODELING FROM IMAGES TAKEN BY A LOW COST CAMERA , 2010 .

[33]  J. Giesen,et al.  Delaunay Triangulation Based Surface Reconstruction: Ideas and Algorithms , 2004 .

[34]  Jonathan Richard Shewchuk.  Stabbing Delaunay Tetrahedralizations , 2004, Discret. Comput. Geom..

[35]  Shi-Min Hu,et al.  Topology Repair of Solid Models Using Skeletons , 2007, IEEE Transactions on Visualization and Computer Graphics.

[36]  Tom Drummond,et al.  ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition , 2009, BMVC.

[37]  Tomáš Svoboda,et al.  Reliable 3D reconstruction from a few catadioptric images , 2002, Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02.

[38]  S. Osher,et al.  Fast surface reconstruction using the level set method , 2001, Proceedings IEEE Workshop on Variational and Level Set Methods in Computer Vision.

[39]  David Israel Lovi,et al.  Incremental free-space carving for real-time 3D reconstruction , 2011 .

[40]  Marshall W. Bern,et al.  Surface Reconstruction by Voronoi Filtering , 1998, SCG '98.

[41]  Michel Dhome,et al.  Generic and real-time structure from motion using local bundle adjustment , 2009, Image Vis. Comput..