Statistical Relational Learning of Grammar Rules for 3D Building Reconstruction

The automatic interpretation of 3D point clouds for building reconstruction is a challenging task. The interpretation process requires highly structured models representing semantics. Formal grammars can describe structures as well as the parameters of buildings and their parts. We propose a novel approach for the automatic learning of weighted attributed context-free grammar rules for 3D building reconstruction, supporting the laborious manual design of rules. We separate structure from parameter learning. Specific Support Vector Machines (SVMs) are used to generate a weighted context-free grammar and predict structured outputs such as parse trees. The grammar is extended by parameters and constraints, which are learned based on a statistical relational learning method using Markov Logic Networks (MLNs). MLNs enforce the topological and geometric constraints. MLNs address uncertainty explicitly and provide probabilistic inference. They are able to deal with partial observations caused by occlusions. Uncertain projective geometry is used to deal with the uncertainty of the observations. Learning is based on a large building database covering different building styles and facade structures. In particular, a treebank that has been derived from the database is employed for structure learning.