Image-based procedural modeling of facades

This paper describes algorithms to automatically derive 3D models of high visual quality from single facade images of arbitrary resolutions. We combine the procedural modeling pipeline of shape grammars with image analysis to derive a meaningful hierarchical facade subdivision. Our system gives rise to three exciting applications: urban reconstruction based on low resolution oblique aerial imagery, reconstruction of facades based on higher resolution ground-based imagery, and the automatic derivation of shape grammar rules from facade images to build a rule base for procedural modeling technology.

[1]  Radomír Mech,et al.  Visual models of plants interacting with their environment , 1996, SIGGRAPH.

[2]  Przemyslaw Prusinkiewicz,et al.  The Algorithmic Beauty of Plants , 1990, The Virtual Laboratory.

[3]  Roberto Cipolla,et al.  Modelling and Interpretation of Architecture from Several Images , 2004, International Journal of Computer Vision.

[4]  Christian Früh,et al.  3D model generation for cities using aerial photographs and ground level laser scans , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[5]  Daniel G. Aliaga,et al.  Build-by-number: rearranging the real world to visualize novel architectural spaces , 2005, VIS 05. IEEE Visualization, 2005..

[6]  Frank Dellaert,et al.  A Probabilistic Approach to the Semantic Interpretation of Building Facades , 2004 .

[7]  William Ribarsky,et al.  From Urban Terrain Models to Visible Cities , 2002, IEEE Computer Graphics and Applications.

[8]  Steven J. Gortler,et al.  Feature-based cellular texturing for architectural models , 2001, SIGGRAPH.

[9]  H. Koning,et al.  The Language of the Prairie: Frank Lloyd Wright's Prairie Houses , 1981 .

[10]  Brendan Lane,et al.  The use of positional information in the modeling of plants , 2001, SIGGRAPH.

[11]  Lu Wang,et al.  Large-Scale Urban Modeling by Combining Ground Level Panoramic and Aerial Imagery , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[12]  Ramakant Nevatia,et al.  Automatic Integration of Facade Textures into 3D Building Models with a Projective Geometry Based Line Clustering , 2002, Comput. Graph. Forum.

[13]  Brendan J. Frey,et al.  Epitomic analysis of appearance and shape , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[14]  Joachim Bauer,et al.  Virtual Habitat: Models of the Urban Outdoors , 2001 .

[15]  U Flemming,et al.  More Than the Sum of Parts: The Grammar of Queen Anne Houses , 1987 .

[16]  Ramakant Nevatia,et al.  Extraction and integration of window in a 3D building model from ground view images , 2004, CVPR 2004.

[17]  Christian Früh,et al.  Constructing 3D city models by merging ground-based and airborne views , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[18]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[19]  Konrad Schindler,et al.  A model-based method for building reconstruction , 2003, First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 2003. HLK 2003..

[20]  Nora Ripperda.  EXTRACTION OF FAÇADES USING RJMCMC AND CONSTRAINT EQUATIONS , 2006 .

[21]  Roberto Cipolla,et al.  Combining Single View Recognition and Multiple View Stereo for Architectural Scenes , 2001, ICCV.

[22]  Ramakant Nevatia,et al.  Interactive 3D building modeling using a hierarchical representation , 2003, First IEEE International Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, 2003. HLK 2003..

[23]  Sven Havemann,et al.  Generative mesh modeling , 2005 .

[24]  Michael Wimmer,et al.  Instant architecture , 2003, ACM Trans. Graph..

[25]  Mark James,et al.  Synthetic topiary , 1994, SIGGRAPH.

[26]  Julien Perret,et al.  The FL-system: a functional L-system for procedural geometric modeling , 2005, The Visual Computer.

[27]  Suya You,et al.  Integrating LiDAR, Aerial Image and Ground Images for Complete Urban Building Modeling , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[28]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, SIGGRAPH 2007.

[29]  William J. Mitchell,et al.  The Palladian Grammar , 1978 .

[30]  Luc Van Gool,et al.  Procedural modeling of buildings , 2006, SIGGRAPH 2006.

[31]  S. Teller,et al.  RECOVERING FACADE TEXTURE AND MICROSTRUCTURE FROM REAL-WORLD IMAGES , 2002 .

[32]  Andrew Zisserman,et al.  Metric rectification for perspective images of planes , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[33]  George Stiny,et al.  Spatial Relations and Grammars , 1982 .

[34]  George Stiny,et al.  Pictorial and Formal Aspects of Shape and Shape Grammars , 1975 .

[35]  Feng Han,et al.  Bottom-up/top-down image parsing by attribute graph grammar , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[36]  Pascal Müller,et al.  Procedural modeling of cities , 2001, SIGGRAPH.

[37]  Paul A. Viola,et al.  Multi-modal volume registration by maximization of mutual information , 1996, Medical Image Anal..

[38]  Robin Liggett,et al.  Virtual Modeling of Urban Environments , 1996, Presence: Teleoperators & Virtual Environments.

[39]  James Arvo,et al.  Creating generative models from range images , 1999, SIGGRAPH.