Guided Real‐Time Scanning of Indoor Objects

Advances in 3D acquisition devices provide unprecedented opportunities for quickly scanning indoor environments. Such raw scans, however, are often noisy, incomplete, and significantly corrupted, making semantic scene understanding difficult, if not impossible. Unfortunately, in most existing workflows, scan quality is assessed after the scanning stage is completed, making it cumbersome to correct for significant missing data by additional scanning. In this work, we present a guided real-time scanning setup, wherein the incoming 3D data stream is continuously analyzed, and the data quality is automatically assessed. While the user is scanning an object, the proposed system discovers and highlights potential missing parts, thus guiding the operator (or an autonomous robot) as where to scan next. The proposed system assesses the quality and completeness of the 3D scan data by comparing to a large collection of commonly occurring indoor man-made objects using an efficient, robust, and effective scan descriptor. We have tested the system on a large number of simulated and real setups, and found the guided interface to be effective even in cluttered and complex indoor environments.

[1]  Niloy J. Mitra,et al.  Symmetry in 3D Geometry: Extraction and Applications , 2013, Comput. Graph. Forum.

[2]  Niloy J. Mitra,et al.  Symmetry in 3D Geometry: Extraction and Applications , 2013, Comput. Graph. Forum.

[3]  G. Burton.  TOPICS IN OPTIMAL TRANSPORTATION (Graduate Studies in Mathematics 58) By CÉDRIC VILLANI: 370 pp., US$59.00, ISBN 0-8218-3312-X (American Mathematical Society, Providence, RI, 2003) , 2004 .

[4]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[5]  Thomas A. Funkhouser,et al.  Shape-based retrieval and analysis of 3d models , 2005, CACM.

[6]  Wolfram Burgard,et al.  Real-time 3D visual SLAM with a hand-held camera , 2011 .

[7]  Andrew E. Johnson,et al.  Spin-Images: A Representation for 3-D Surface Matching , 1997 .

[8]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[9]  Dieter Fox,et al.  Interactive 3D modeling of indoor environments with a consumer depth camera , 2011, UbiComp '11.

[10]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[11]  Sebastian Thrun,et al.  Shape from symmetry , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[12]  Vincent Lepetit,et al.  Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes , 2011, 2011 International Conference on Computer Vision.

[13]  In Kyu Park,et al.  Content-based 3D model retrieval using a single depth image from a low-cost 3D camera , 2013, The Visual Computer.

[14]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..

[15]  Leonidas J. Guibas,et al.  Acquiring 3D indoor environments with variability and repetition , 2012, ACM Trans. Graph..

[16]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[17]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Roland Siegwart,et al.  Segmentation and Unsupervised Part-based Discovery of Repetitive Objects , 2010, Robotics: Science and Systems.

[19]  Takeo Kanade,et al.  Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces , 2010, NIPS.

[20]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[21]  Stephen DiVerdi,et al.  Learning part-based templates from large collections of 3D shapes , 2013, ACM Trans. Graph..

[22]  C. Villani.  Topics in Optimal Transportation , 2003 .

[23]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[24]  Kun Zhou,et al.  Interactive images , 2012, ACM Trans. Graph..

[25]  Leonidas J. Guibas,et al.  Discovering structural regularity in 3D geometry , 2008, SIGGRAPH 2008.

[26]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[27]  Silvio Savarese,et al.  Estimating the aspect layout of object categories , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Hossein Ebrahimnezhad,et al.  3D Object Categorization Based on Histogram of Distance and Normal Vector Angles on Surface Points , 2011, 2011 7th Iranian Conference on Machine Vision and Image Processing.

[29]  Dieter Fox,et al.  RGB-(D) scene labeling: Features and algorithms , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Young Min Kim,et al.  Interactive acquisition of residential floor plans , 2012, 2012 IEEE International Conference on Robotics and Automation.

[31]  Ioannis Pratikakis,et al.  3D object retrieval via range image queries in a bag-of-visual-words context , 2013, The Visual Computer.

[32]  Thorsten Joachims,et al.  Semantic Labeling of 3D Point Clouds for Indoor Scenes , 2011, NIPS.

[33]  Leonidas J. Guibas,et al.  Example-Based 3D Scan Completion , 2005 .

[34]  Hans-Peter Seidel,et al.  Exploring Shape Variations by 3D‐Model Decomposition and Part‐based Recombination , 2012, Comput. Graph. Forum.

[35]  Daniel Cohen-Or,et al.  Structure-aware shape processing , 2013, Eurographics.

[36]  Ioannis Pratikakis,et al.  3D Object Retrieval via Range Image Queries based on SIFT descriptors on Panoramic Views , 2012, 3DOR@Eurographics.

[37]  Ke Xie,et al.  A search-classify approach for cluttered indoor scene understanding , 2012, ACM Trans. Graph..