A Survey of Simple Geometric Primitives Detection Methods for Captured 3D Data

The amount of captured 3D data is continuously increasing, with the democratization of consumer depth cameras, the development of modern multi‐view stereo capture setups and the rise of single‐view 3D capture based on machine learning. The analysis and representation of this ever growing volume of 3D data, often corrupted with acquisition noise and reconstruction artefacts, is a serious challenge at the frontier between computer graphics and computer vision. To that end, segmentation and optimization are crucial analysis components of the shape abstraction process, which can themselves be greatly simplified when performed on lightened geometric formats. In this survey, we review the algorithms which extract simple geometric primitives from raw dense 3D data. After giving an introduction to these techniques, from the acquisition modality to the underlying theoretical concepts, we propose an application‐oriented characterization, designed to help select an appropriate method based on one's application needs and compare recent approaches. We conclude by giving hints for how to evaluate these methods and a set of research challenges to be explored.

[1]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[2]  Ariel Shamir,et al.  A survey on Mesh Segmentation Techniques , 2008, Comput. Graph. Forum.

[3]  Reinhard Klein,et al.  Towards the Extraction of Hierarchical Building Descriptions from 3D Indoor Scans , 2014, 3DOR@Eurographics.

[4]  Tamy Boubekeur,et al.  Sphere-Meshes , 2013, ACM Trans. Graph..

[5]  Jie Chen,et al.  Architectural Modeling from Sparsely Scanned Range Data , 2008, International Journal of Computer Vision.

[6]  Ralph R. Martin,et al.  Robust Segmentation of Primitives from Range Data in the Presence of Geometric Degeneracy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Reinhard Klein,et al.  Shape Recognition in 3D Point-Clouds , 2008 .

[8]  Kun Zhou,et al.  An interactive approach to semantic modeling of indoor scenes with an RGBD camera , 2012, ACM Trans. Graph..

[9]  Sven Oesau,et al.  Planar Shape Detection and Regularization in Tandem , 2016, Comput. Graph. Forum.

[10]  Ross T. Whitaker,et al.  Indoor Scene Reconstruction from Sets of Noisy Range Images , 2001, Graph. Model..

[11]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[12]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[13]  Mohammed Bennamoun,et al.  Separating objects and clutter in indoor scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  George Vosselman,et al.  An integrated approach for modelling and global registration of point clouds , 2007 .

[15]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jiri Matas,et al.  Randomized RANSAC with Td, d test , 2004, Image Vis. Comput..

[17]  Shimin Hu,et al.  Adaptive Partitioning of Urban Facades , 2011 .

[18]  Michael Wimmer,et al.  O-snap , 2013, ACM Trans. Graph..

[19]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[20]  Vladimir G. Kim,et al.  Data‐Driven Shape Analysis and Processing , 2015, Comput. Graph. Forum.

[21]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[22]  Se-Young Oh,et al.  Indoor mapping using planes extracted from noisy RGB-D sensors , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Dong-Ming Yan,et al.  Variational mesh segmentation via quadric surface fitting , 2012, Comput. Aided Des..

[24]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[25]  Kun Zhou,et al.  Online Structure Analysis for Real-Time Indoor Scene Reconstruction , 2015, ACM Trans. Graph..

[26]  Guannan Gao,et al.  Probabilistic Hough Transform , 2011 .

[27]  Suyash P. Awate,et al.  Computer Vision, Graphics, and Image Processing , 2016, Lecture Notes in Computer Science.

[28]  S. D. Wang,et al.  SEMI-AUTOMATED CSG MODEL-BASED BUILDING EXTRACTION FROM PHOTOGRAMMETRIC IMAGES , 2004 .

[29]  Florent Lafarge,et al.  Creating Large-Scale City Models from 3D-Point Clouds: A Robust Approach with Hybrid Representation , 2012, International Journal of Computer Vision.

[30]  Chen Feng,et al.  Point-plane SLAM for hand-held 3D sensors , 2013, 2013 IEEE International Conference on Robotics and Automation.

[31]  Ann Patricia Fothergill,et al.  Forming Models Of Plane-And-Cylinder Faceled Bodies From Light Stripes , 1975, IJCAI.

[32]  Pierre Alliez,et al.  Variational shape approximation , 2004, ACM Trans. Graph..

[33]  Marco Attene,et al.  Hierarchical Structure Recovery of Point‐Sampled Surfaces , 2010, Comput. Graph. Forum.

[34]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[35]  Florent Lafarge,et al.  Surface Reconstruction through Point Set Structuring , 2013, Comput. Graph. Forum.

[36]  Daniel Cohen-Or,et al.  Generalized cylinder decomposition , 2015, ACM Trans. Graph..

[37]  Stephen DiVerdi,et al.  Learning part-based templates from large collections of 3D shapes , 2013, ACM Trans. Graph..

[38]  A. M. Andrew,et al.  Another Efficient Algorithm for Convex Hulls in Two Dimensions , 1979, Inf. Process. Lett..

[39]  M. Veloso,et al.  Planar polygon extraction and merging from depth images , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Henrik I. Christensen,et al.  Efficient Organized Point Cloud Segmentation with Connected Components , 2013 .

[42]  Hanns-F. Schuster SEGMENTATION OF LIDAR DATA USING THE TENSOR VOTING FRAMEWORK , 2004 .

[43]  Pascal Fua,et al.  Probability occupancy maps for occluded depth images , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Leif Kobbelt,et al.  Structure Recovery via Hybrid Variational Surface Approximation , 2005, Comput. Graph. Forum.

[45]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Jiajun Zhang,et al.  Extraction of cylinders and cones from minimal point sets , 2016, Graph. Model..

[47]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[48]  Chao-Hung Lin,et al.  Point cloud modeling using algebraic template , 2011 .

[49]  Manuela Veloso,et al.  Fast Sampling Plane Filtering , Polygon Construction and Merging from Depth Images , 2011 .

[50]  Sven Loncaric,et al.  A survey of shape analysis techniques , 1998, Pattern Recognit..

[51]  Björn Stenger,et al.  Contraction Moves for Geometric Model Fitting , 2012, ECCV.

[52]  Daniel Cohen-Or,et al.  3-Sweep , 2013, ACM Trans. Graph..

[53]  Daniel Cohen-Or,et al.  Geosemantic Snapping for Sketch‐Based Modeling , 2013, Comput. Graph. Forum.

[54]  Ahad Harati,et al.  Parallel RANSAC: Speeding up plane extraction in RGBD image sequences using GPU , 2015, 2015 5th International Conference on Computer and Knowledge Engineering (ICCKE).

[55]  N. Mitra,et al.  GlobFit: consistently fitting primitives by discovering global relations , 2011, SIGGRAPH 2011.

[56]  Vineet R. Kamat,et al.  Fast plane extraction in organized point clouds using agglomerative hierarchical clustering , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[57]  Serge Beucher,et al.  Use of watersheds in contour detection , 1979 .

[58]  Marc Alexa,et al.  Computing and Rendering Point Set Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[59]  Junhao Xiao,et al.  3D point cloud registration based on planar surfaces , 2012, 2012 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[60]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[61]  Paul H. J. Kelly,et al.  Dense planar SLAM , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[62]  Manuel Menezes de Oliveira Neto,et al.  Improved Scene Reconstruction from Range Images , 2002, Comput. Graph. Forum.

[63]  Nico Blodow,et al.  Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[64]  Roland Siegwart,et al.  3D SLAM using planar segments , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[65]  Andrew Owens,et al.  SUN3D: A Database of Big Spaces Reconstructed Using SfM and Object Labels , 2013, 2013 IEEE International Conference on Computer Vision.

[66]  Nico Blodow,et al.  Towards 3D object maps for autonomous household robots , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[67]  Bruno Lévy,et al.  Least squares conformal maps for automatic texture atlas generation , 2002, ACM Trans. Graph..

[68]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[69]  Luc Van Gool,et al.  3D all the way: Semantic segmentation of urban scenes from start to end in 3D , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[70]  T. Rabbani,et al.  EFFICIENT HOUGH TRANSFORM FOR AUTOMATIC DETECTION OF CYLINDERS IN POINT CLOUDS , 2005 .

[71]  Thomas Funkhouser,et al.  A benchmark for 3D mesh segmentation , 2009, SIGGRAPH 2009.

[72]  Lucian Cosmin Goron,et al.  Robustly Segmenting Cylindrical and Box-like Objects in Cluttered Scenes using Depth Cameras , 2012, ROBOTIK.

[73]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[74]  Manuel Menezes de Oliveira Neto,et al.  Real-time detection of planar regions in unorganized point clouds , 2015, Pattern Recognit..

[75]  Leonidas J. Guibas,et al.  Shape segmentation using local slippage analysis , 2004, SGP '04.

[76]  Hungwen Li,et al.  Fast Hough transform: A hierarchical approach , 1986, Comput. Vis. Graph. Image Process..

[77]  Kun Zhou,et al.  Real-time soft shadows in dynamic scenes using spherical harmonic exponentiation , 2006, ACM Trans. Graph..

[78]  Marshall W. Bern,et al.  Surface Reconstruction by Voronoi Filtering , 1998, SCG '98.

[79]  Henrik I. Christensen,et al.  Planar surface SLAM with 3D and 2D sensors , 2012, 2012 IEEE International Conference on Robotics and Automation.

[80]  Martial Hebert,et al.  Natural terrain classification using three‐dimensional ladar data for ground robot mobility , 2006, J. Field Robotics.

[81]  Kang Chen,et al.  Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information , 2014, ACM Trans. Graph..

[82]  Rida T. Farouki,et al.  Approximation of rolling-ball blends for free-form parametric surfaces , 1996, Comput. Aided Des..

[83]  P. Rousseeuw Least Median of Squares Regression , 1984 .

[84]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[85]  George Vosselman,et al.  3D BUILDING MODEL RECONSTRUCTION FROM POINT CLOUDS AND GROUND PLANS , 2001 .

[86]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[87]  Yaser Sheikh,et al.  Monocular Object Detection Using 3D Geometric Primitives , 2012, ECCV.

[88]  Yoshiaki Shirai,et al.  Recognition of polyhedrons with a range finder , 1971, IJCAI.

[89]  Zhen Li,et al.  Primitive Fitting Based on the Efficient multiBaySAC Algorithm , 2015, PloS one.

[90]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[91]  Manuel Menezes de Oliveira Neto,et al.  Real-time line detection through an improved Hough transform voting scheme , 2008, Pattern Recognit..

[92]  Pierre Alliez,et al.  State of the Art in Surface Reconstruction from Point Clouds , 2014, Eurographics.

[93]  Michael Garland,et al.  Surface simplification using quadric error metrics , 1997, SIGGRAPH.

[94]  Niloy J. Mitra,et al.  RAPter , 2015, ACM Trans. Graph..

[95]  ARISTIDES A. G. REQUICHA,et al.  Representations for Rigid Solids: Theory, Methods, and Systems , 1980, CSUR.

[96]  Mostafa Abdel-Bary,et al.  3D Laser Scanners’ Techniques Overview , 2015 .

[97]  Jianxiong Xiao,et al.  A Linear Approach to Matching Cuboids in RGBD Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[98]  Tamy Boubekeur,et al.  Animated Mesh Approximation With Sphere-Meshes , 2016, ACM Trans. Graph..

[99]  Pavel Smrz,et al.  Continuous plane detection in point-cloud data based on 3D Hough Transform , 2014, J. Vis. Commun. Image Represent..

[100]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[101]  William Puech,et al.  Recovering primitives in 3D CAD meshes , 2011, Electronic Imaging.

[102]  Andrea Tagliasacchi,et al.  Sphere-meshes for real-time hand modeling and tracking , 2016, ACM Trans. Graph..

[103]  Björn Stenger,et al.  Demisting the Hough Transform for 3D Shape Recognition and Registration , 2014, International Journal of Computer Vision.

[104]  Olga Sorkine-Hornung,et al.  Object detection and classification from large‐scale cluttered indoor scans , 2014, Comput. Graph. Forum.

[105]  H. P. Hildre,et al.  Fast plane detection for SLAM from noisy range images in both structured and unstructured environments , 2011, 2011 IEEE International Conference on Mechatronics and Automation.

[106]  Franco P. Preparata,et al.  On circular Cylinders by Four or Five Points in Space , 2001 .

[107]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[108]  Florent Lafarge,et al.  LOD Generation for Urban Scenes , 2015, ACM Trans. Graph..

[109]  Sunghee Choi,et al.  The power crust , 2001, SMA '01.

[110]  Karan Singh,et al.  Extraction and remeshing of ellipsoidal representations from mesh data , 2005, Graphics Interface.

[111]  Richard S. Stephens,et al.  Probabilistic approach to the Hough transform , 1991, Image Vis. Comput..

[112]  Sven Behnke,et al.  Real-Time Plane Segmentation Using RGB-D Cameras , 2012, RoboCup.

[113]  Hui Lin,et al.  Semantic decomposition and reconstruction of residential scenes from LiDAR data , 2013, ACM Trans. Graph..

[114]  Erkki Oja,et al.  A new curve detection method: Randomized Hough transform (RHT) , 1990, Pattern Recognit. Lett..

[115]  Frédo Durand,et al.  Billboard clouds for extreme model simplification , 2003, ACM Trans. Graph..

[116]  Thomas O. Binford,et al.  Computer Description of Curved Objects , 1973, IEEE Transactions on Computers.

[117]  Hans-Peter Seidel,et al.  Multi-level partition of unity implicits , 2003, ACM Trans. Graph..

[118]  Manuela M. Veloso,et al.  Depth camera based indoor mobile robot localization and navigation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[119]  Reinhard Klein,et al.  Automatic reconstruction of parametric building models from indoor point clouds , 2016, Comput. Graph..

[120]  Daniel Cohen-Or,et al.  GlobFit: consistently fitting primitives by discovering global relations , 2011, ACM Trans. Graph..

[121]  El-Houssine Bouyakhf,et al.  Planes Detection for Robust Localization and Mapping in RGB-D SLAM Systems , 2015, 2015 International Conference on 3D Vision.

[122]  Marco Attene,et al.  Hierarchical mesh segmentation based on fitting primitives , 2006, The Visual Computer.

[123]  Michael Kaess,et al.  Simultaneous localization and mapping with infinite planes , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[124]  Olivier D. Faugeras,et al.  A 3-D Recognition and Positioning Algorithm Using Geometrical Matching Between Primitive Surfaces , 1983, IJCAI.

[125]  Paolo Cignoni,et al.  Metro: Measuring Error on Simplified Surfaces , 1998, Comput. Graph. Forum.

[126]  Ralph R. Martin,et al.  Faithful Least-Squares Fitting of Spheres, Cylinders, Cones and Tori for Reliable Segmentation , 1998, ECCV.