Segmentation of building facades using procedural shape priors

In this paper we propose a novel approach to the perceptual interpretation of building facades that combines shape grammars, supervised classification and random walks. Procedural modeling is used to model the geometric and the photometric variation of buildings. This is fused with visual classification techniques (randomized forests) that provide a crude probabilistic interpretation of the observation space in order to measure the appropriateness of a procedural generation with respect to the image. A random exploration of the grammar space is used to optimize the sequence of derivation rules towards a semantico-geometric interpretation of the observations. Experiments conducted on complex architecture facades with ground truth validate the approach.

[1]  J. Gips.  Shape grammars and their uses , 1974 .

[2]  George Stiny,et al.  Pictorial and Formal Aspects of Shape and Shape Grammars , 1975 .

[3]  Timothy F. Cootes,et al.  Active shape models , 1998 .

[4]  C. Taylor,et al.  Active shape models - 'Smart Snakes'. , 1992 .

[5]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[7]  Pascal Müller,et al.  Procedural modeling of cities , 2001, SIGGRAPH.

[8]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  S. Osher,et al.  Geometric Level Set Methods in Imaging, Vision, and Graphics , 2011, Springer New York.

[11]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[12]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Frank Dellaert,et al.  A Probabilistic Approach to the Semantic Interpretation of Building Facades , 2004 .

[14]  R. Zabih,et al.  What energy functions can be minimized via graph cuts , 2004 .

[15]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[16]  Luc Van Gool,et al.  Procedural modeling of buildings , 2006, ACM Trans. Graph..

[17]  Vincent Lepetit,et al.  Keypoint recognition using randomized trees , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jamie Shotton,et al.  The Layout Consistent Random Field for Recognizing and Segmenting Partially Occluded Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[19]  Aaron F. Bobick,et al.  Multiscale 3-D Shape Representation and Segmentation Using Spherical Wavelets , 2007, IEEE Transactions on Medical Imaging.

[20]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, SIGGRAPH 2007.

[21]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Georgios Tziritas,et al.  Single view reconstruction using shape grammars for urban environments , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  Nikos Paragios,et al.  Hierarchical 3D diffusion wavelet shape priors , 2009, 2009 IEEE 12th International Conference on Computer Vision.