Insertion of 3-D-Primitives in Mesh-Based Representations: Towards Compact Models Preserving the Details

We propose an original hybrid modeling process of urban scenes that represents 3-D models as a combination of mesh-based surfaces and geometric 3-D-primitives. Meshes describe details such as ornaments and statues, whereas 3-D-primitives code for regular shapes such as walls and columns. Starting from an 3-D-surface obtained by multiview stereo techniques, these primitives are inserted into the surface after being detected. This strategy allows the introduction of semantic knowledge, the simplification of the modeling, and even correction of errors generated by the acquisition process. We design a hierarchical approach exploring different scales of an observed scene. Each level consists first in segmenting the surface using a multilabel energy model optimized by -expansion and then in fitting 3-D-primitives such as planes, cylinders or tori on the obtained partition where relevant. Experiments on real meshes, depth maps and synthetic surfaces show good potential for the proposed approach.

[1]  ZhaoPeng,et al.  Image-based faade modeling , 2008 .

[2]  James F. O'Brien,et al.  Interpolating and approximating implicit surfaces from polygon soup , 2005, SIGGRAPH Courses.

[3]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[4]  Richard Szeliski,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, SIGGRAPH Asia '08.

[5]  Ariel Shamir,et al.  A survey on Mesh Segmentation Techniques , 2008, Comput. Graph. Forum.

[6]  Helmut Mayer,et al.  Object extraction in photogrammetric computer vision , 2008 .

[7]  Leif Kobbelt,et al.  Structure Recovery via Hybrid Variational Surface Approximation , 2005, Comput. Graph. Forum.

[8]  Ramakant Nevatia,et al.  Extraction and integration of window in a 3D building model from ground view images , 2004, CVPR 2004.

[9]  Nora Ripperda EXTRACTION OF FAÇADES USING RJMCMC AND CONSTRAINT EQUATIONS , 2006 .

[10]  Feng Han,et al.  Bottom-up/top-down image parsing by attribute graph grammar , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[11]  Horst Bischof,et al.  Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial Imagery , 2008, ECCV.

[12]  Florent Lafarge,et al.  Combining Meshes and Geometric Primitives for Accurate and Semantic Modeling , 2009, BMVC.

[13]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[14]  Dong-Ming Yan,et al.  Quadric Surface Extraction by Variational Shape Approximation , 2006, GMP.

[15]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[17]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Andrew Blake,et al.  LogCut - Efficient Graph Cut Optimization for Markov Random Fields , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Mathieu Desbrun,et al.  Variational shape approximation , 2004, SIGGRAPH 2004.

[20]  H. Hirschmüller Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Stereo Processing by Semi-global Matching and Mutual Information , 2022 .

[21]  Ramakant Nevatia,et al.  Extraction and integration of window in a 3D building model from ground view images , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[22]  Andrew Zisserman,et al.  Automatic reconstruction of piecewise planar models from multiple views , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[23]  Marco Attene,et al.  Mesh Segmentation - A Comparative Study , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[24]  Takeo Kanade,et al.  Modeling and Representations of Large-Scale 3D Scenes , 2008, International Journal of Computer Vision.

[25]  Ronald Fedkiw,et al.  Level set methods and dynamic implicit surfaces , 2002, Applied mathematical sciences.

[26]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, SIGGRAPH 2007.

[28]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Chitra Dorai,et al.  COSMOS - A Representation Scheme for 3D Free-Form Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[32]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[33]  K JainAnil,et al.  COSMOS-A Representation Scheme for 3D Free-Form Objects , 1997 .

[34]  Pascal Fua,et al.  Implicit meshes for surface reconstruction , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Suya You,et al.  Automatic reconstruction of cities from remote sensor data , 2009, CVPR.

[36]  Marc Pollefeys,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, SIGGRAPH 2008.

[37]  Ralph R. Martin,et al.  Robust Segmentation of Primitives from Range Data in the Presence of Geometric Degeneracy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Zhuowen Tu,et al.  Range image segmentation by an effective jump-diffusion method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[40]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[41]  Leonidas J. Guibas,et al.  Discovering structural regularity in 3D geometry , 2008, ACM Trans. Graph..

[42]  Anders Heyden,et al.  Convex multi-region segmentation on manifolds , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[43]  Roberto Cipolla,et al.  Modelling and Interpretation of Architecture from Several Images , 2004, International Journal of Computer Vision.

[44]  Jean-Philippe Pons,et al.  Hierarchical shape-based surface reconstruction for dense multi-view stereo , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[45]  Rachid Deriche,et al.  RECONSTRUCTION OF 3 D LINEAR PRIMITIVES FROM MULTIPLE VIEWS FOR URBAN AREAS MODELISATION , 2002 .

[46]  Emmanuel P. Baltsavias,et al.  Object extraction and revision by image analysis using existing geodata and knowledge: current status and steps towards operational systems☆ , 2004 .

[47]  Vivek Verma,et al.  3D Building Detection and Modeling from Aerial LIDAR Data , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[48]  Christian Früh,et al.  An Automated Method for Large-Scale, Ground-Based City Model Acquisition , 2004, International Journal of Computer Vision.

[49]  Jean-Philippe Pons,et al.  Transductive Segmentation of Textured Meshes , 2009, ACCV.

[50]  Marc Pierrot Deseilligny,et al.  3D Building Reconstruction with Parametric Roof Superstructures , 2007, 2007 IEEE International Conference on Image Processing.

[51]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Takeshi Oishi,et al.  Flying Laser Range Sensor for Large-Scale Site-Modeling and Its Applications in Bayon Digital Archival Project , 2008, International Journal of Computer Vision.

[53]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[54]  Josiane Zerubia,et al.  Building reconstruction from a single DEM , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[56]  Ulrich Neumann,et al.  A streaming framework for seamless building reconstruction from large-scale aerial LiDAR data , 2009, CVPR.

[57]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[58]  Jianxiong Xiao,et al.  Image-based façade modeling , 2008, SIGGRAPH 2008.