Template Assembly for Detailed Urban Reconstruction

We propose a new framework to reconstruct building details by automatically assembling 3D templates on coarse textured building models. In a preprocessing step, we generate an initial coarse model to approximate a point cloud computed using Structure from Motion and Multi View Stereo, and we model a set of 3D templates of facade details. Next, we optimize the initial coarse model to enforce consistency between geometry and appearance (texture images). Then, building details are reconstructed by assembling templates on the textured faces of the coarse model. The 3D templates are automatically chosen and located by our optimization‐based template assembly algorithm that balances image matching and structural regularity. In the results, we demonstrate how our framework can enrich the details of coarse models using various data sets.

[1]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[2]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[3]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[4]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[5]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[7]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Roberto Cipolla,et al.  Modelling and Interpretation of Architecture from Several Images , 2004, International Journal of Computer Vision.

[9]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[10]  Jianxiong Xiao,et al.  Image-based façade modeling , 2008, ACM Trans. Graph..

[11]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, SIGGRAPH 2010.

[12]  Horst Bischof,et al.  Fusion of Feature- and Area-Based Information for Urban Buildings Modeling from Aerial Imagery , 2008, ECCV.

[13]  D. Cohen-Or,et al.  SmartBoxes for interactive urban reconstruction , 2010, ACM Trans. Graph..

[14]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Ulrich Neumann,et al.  2.5D building modeling by discovering global regularities , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Yanxi Liu,et al.  Translation-Symmetry-Based Perceptual Grouping with Applications to Urban Scenes , 2010, ACCV.

[17]  Marc Pollefeys,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, SIGGRAPH 2008.

[18]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[19]  Rafael Grompone von Gioi,et al.  LSD: A Fast Line Segment Detector with a False Detection Control , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Tai-Pang Wu,et al.  Quasi-dense 3D reconstruction using tensor-based multiview stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[22]  Niloy J. Mitra,et al.  Coupled structure-from-motion and 3D symmetry detection for urban facades , 2014, ACM Trans. Graph..

[23]  Hui Lin,et al.  Semantic decomposition and reconstruction of residential scenes from LiDAR data , 2013, ACM Trans. Graph..

[24]  Shi-Min Hu,et al.  Adaptive partitioning of urban facades , 2011, SA '11.

[25]  Daniel G. Aliaga,et al.  Building reconstruction using manhattan-world grammars , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Shimin Hu,et al.  Adaptive Partitioning of Urban Facades , 2011 .

[27]  Daniel G. Aliaga,et al.  Modeling the Appearance and Behavior of Urban Spaces , 2009, Eurographics.

[28]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[30]  Andrew Zisserman,et al.  Metric rectification for perspective images of planes , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[31]  Steven M. Seitz,et al.  Schematic surface reconstruction , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Nancy Argüelles,et al.  Author ' s , 2008 .

[33]  Michael Wimmer,et al.  O-snap , 2013, ACM Trans. Graph..

[34]  Iasonas Kokkinos,et al.  Parsing Facades with Shape Grammars and Reinforcement Learning , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Ulrich Neumann,et al.  Complete residential urban area reconstruction from dense aerial LiDAR point clouds , 2013, Graph. Model..

[36]  Jianxiong Xiao,et al.  Image-based street-side city modeling , 2009, ACM Trans. Graph..

[37]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, ACM Trans. Graph..

[38]  Andrew Zisserman,et al.  New Techniques for Automated Architectural Reconstruction from Photographs , 2002, ECCV.

[39]  Richard Szeliski,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, ACM Trans. Graph..

[40]  Daniel G. Aliaga,et al.  Automatic Extraction of Manhattan-World Building Masses from 3D Laser Range Scans , 2012, IEEE Transactions on Visualization and Computer Graphics.

[41]  Daniel Cohen-Or,et al.  GlobFit: consistently fitting primitives by discovering global relations , 2011, ACM Trans. Graph..

[42]  Jan-Michael Frahm,et al.  Detailed Real-Time Urban 3D Reconstruction from Video , 2007, International Journal of Computer Vision.

[43]  Jan-Michael Frahm,et al.  Repetition-based dense single-view reconstruction , 2011, CVPR 2011.

[44]  Daniel G. Aliaga,et al.  A Survey of Urban Reconstruction , 2013, Comput. Graph. Forum.

[45]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, CVPR.

[46]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, ACM Trans. Graph..