Manhattan and Piecewise-Planar Constraints for Dense Monocular Mapping

This paper presents a variational formulation for real-time dense 3D mapping from a RGB monocular sequence that incorporates Manhattan and piecewise-planar constraints in indoor and outdoor man-made scenes. The state-of-the-art variational approaches are based on the minimization of an energy functional composed of two terms, the first one accounting for the photometric compatibility in multiple views, and the second one favoring smooth solutions. We show that the addition of a third energy term modelling Manhattan and piecewise-planar structures greatly improves the accuracy of the dense visual maps, particularly for low-textured man-made environments where the data term can be ambiguous. We evaluate two different methods to provide such Manhattan and piecewise-planar constraints based on 1) multiview superpixel geometry and 2) multiview layout estimation and scene understanding. Our experiments include the largest map produced by variational methods from a RGB sequence and demonstrate a reduction in the median depth error up to a factor 5×.