Structural Modeling from Depth Images

In this work, we present a new automatic system for scene reconstruction of high-level structural models. We start with identifying planar regions in depth images obtained with a SLAM system. Our main contribution is an approach which identifies constraints such as incidence and orthogonality of planar surfaces and uses them in an incremental optimization framework to extract high-level structural models. The result is a manifold mesh with a low number of polygons, immediately useful in many Augmented Reality applications such as inspection, interior design or spatial interaction.

[1]  Michael Wimmer,et al.  O-snap , 2013, ACM Trans. Graph..

[2]  Philip H. S. Torr,et al.  VideoTrace: rapid interactive scene modelling from video , 2007, SIGGRAPH 2007.

[3]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Anton van den Hengel,et al.  In situ image-based modeling , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[5]  Dieter Fox,et al.  Detection-based object labeling in 3D scenes , 2012, 2012 IEEE International Conference on Robotics and Automation.

[6]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[7]  Alexei A. Efros,et al.  Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics , 2010, ECCV.

[8]  Young Min Kim,et al.  Interactive acquisition of residential floor plans , 2012, 2012 IEEE International Conference on Robotics and Automation.

[9]  Paul H. J. Kelly,et al.  SLAM++: Simultaneous Localisation and Mapping at the Level of Objects , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Paul H. J. Kelly,et al.  Dense planar SLAM , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[11]  Peter H. N. de With,et al.  Incremental and batch planar simplification of dense point cloud maps , 2015, Robotics Auton. Syst..

[12]  Dieter Schmalstieg,et al.  Interactive syntactic modeling with a single-point laser range finder and camera , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[13]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[14]  Reinhard Klein,et al.  Completion and Reconstruction with Primitive Shapes , 2009, Comput. Graph. Forum.

[15]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..

[16]  Frank Dellaert,et al.  Planar Segmentation of RGBD Images Using Fast Linear Fitting and Markov Chain Monte Carlo , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[17]  Pished Bunnun,et al.  OutlinAR: an assisted interactive model building system with reduced computational effort , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[18]  Philip H. S. Torr,et al.  VideoTrace: rapid interactive scene modelling from video , 2007, ACM Trans. Graph..

[19]  Wolfram Burgard,et al.  G2o: A general framework for graph optimization , 2011, 2011 IEEE International Conference on Robotics and Automation.

[20]  Takeo Kanade,et al.  Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces , 2010, NIPS.

[21]  Andrew J. Davison,et al.  A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Walterio W. Mayol-Cuevas,et al.  Ninja on a Plane: Automatic Discovery of Physical Planes for Augmented Reality Using Visual SLAM , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[23]  Daniel Cohen-Or,et al.  GlobFit: consistently fitting primitives by discovering global relations , 2011, ACM Trans. Graph..

[24]  Richard Szeliski,et al.  Interactive 3D architectural modeling from unordered photo collections , 2008, ACM Trans. Graph..

[25]  Henrik I. Christensen,et al.  Planar surface SLAM with 3D and 2D sensors , 2012, 2012 IEEE International Conference on Robotics and Automation.

[26]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.