Quasi-regular Facade Structure Extraction

In this paper we present a novel two-stage framework for extracting what we define as a quasi-regular structure in facade images. A quasi-regular structure is an irregular rectangular grid representing the placements of repetitive structural architecture objects, e.g., windows, in a facade. Such a structure generalizes a perfect lattice structure generated by the 2D symmetry groups, studied by the previous work. First, we propose to formulate the quasi-regular structure detection in an object-oriented Marked Point Process framework by treating the architectural elements as objects. This leads to an initial quasi-regular structure map which serves as an indicator map of potential object locations. Then, we propose a regularization scheme to recover the complete quasi-regular structures from the initial incomplete structure. This stage takes advantage of the intrinsic low rank constraint of the quasi-regular structure representing a regularized facade. By applying such a regularization, the complete quasi-regular facade structure is obtained. We have extensively tested our method on a large variety of facade images, and demonstrated both the effectiveness and the robustness of our two-stage framework.

[1]  Jianxiong Xiao,et al.  Image-based façade modeling , 2008, ACM Trans. Graph..

[2]  Radim Sára,et al.  Modeling symmetries for stochastic structural recognition , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[3]  Xavier Descombes,et al.  Stochastic geometry for image analysis , 2011 .

[4]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[5]  Yi Ma,et al.  TILT: Transform Invariant Low-Rank Textures , 2010, ACCV.

[6]  Chao Yang,et al.  Parsing façade with rank-one approximation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Radim Sára,et al.  A Weak Structure Model for Regular Pattern Recognition Applied to Facade Images , 2010, ACCV.

[8]  Josiane Zerubia,et al.  Building extraction from digital elevation models , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[9]  Luc Van Gool,et al.  Image-based procedural modeling of facades , 2007, ACM Trans. Graph..

[10]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[11]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[12]  Yi Ma,et al.  TILT: Transform Invariant Low-Rank Textures , 2010, ACCV 2010.

[13]  Luc Van Gool,et al.  Computational Symmetry in Computer Vision and Computer Graphics , 2010, Found. Trends Comput. Graph. Vis..

[14]  Yanxi Liu,et al.  Translation-Symmetry-Based Perceptual Grouping with Applications to Urban Scenes , 2010, ACCV.

[15]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[16]  Frank Dellaert,et al.  A Probabilistic Approach to the Semantic Interpretation of Building Facades , 2004 .

[17]  Josiane Zerubia,et al.  Building Outline Extraction from Digital Elevation Models Using Marked Point Processes , 2007, International Journal of Computer Vision.

[18]  Tom Drummond,et al.  Faster and Better: A Machine Learning Approach to Corner Detection , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Hans-Peter Seidel,et al.  Symmetry Detection Using Feature Lines , 2009, Comput. Graph. Forum.

[20]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[21]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[22]  Hayko Riemenschneider,et al.  Irregular lattices for complex shape grammar facade parsing , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Andrew W. Fitzgibbon,et al.  Direct Least Square Fitting of Ellipses , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Yanxi Liu,et al.  Detecting and matching repeated patterns for automatic geo-tagging in urban environments , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[26]  Nikos Paragios,et al.  Segmentation of building facades using procedural shape priors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[28]  Antti Oulasvirta,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[29]  Long Quan,et al.  Translation symmetry detection in a fronto-parallel view , 2011, CVPR 2011.

[30]  Chun Liu,et al.  Image-based Modeling of Haussmannian Facades , 2010 .

[31]  Iasonas Kokkinos,et al.  Shape grammar parsing via Reinforcement Learning , 2011, CVPR 2011.

[32]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Andrew W. Fitzgibbon,et al.  Direct least squares fitting of ellipses , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[34]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[36]  Nora Ripperda.  Grammar Based Facade Reconstruction using rjMCMC , 2008 .

[37]  Christopher Rasmussen,et al.  2D Lattice Extraction from Structured Environments , 2007, 2007 IEEE International Conference on Image Processing.