H-RANSAC: A HYBRID POINT CLOUD SEGMENTATION COMBINING 2D AND 3D DATA

Abstract. In this paper, we present a novel 3D segmentation approach operating on point clouds generated from overlapping images. The aim of the proposed hybrid approach is to effectively segment co-planar objects, by leveraging the structural information originating from the 3D point cloud and the visual information from the 2D images, without resorting to learning based procedures. More specifically, the proposed hybrid approach, H-RANSAC, is an extension of the well-known RANSAC plane-fitting algorithm, incorporating an additional consistency criterion based on the results of 2D segmentation. Our expectation that the integration of 2D data into 3D segmentation will achieve more accurate results, is validated experimentally in the domain of 3D city models. Results show that HRANSAC can successfully delineate building components like main facades and windows, and provide more accurate segmentation results compared to the typical RANSAC plane-fitting algorithm.

[1]  Andrés Serna,et al.  Segmentation and Classification of 3D Urban Point Clouds: Comparison and Combination of Two Approaches , 2015, FSR.

[2]  Reinhard Klein,et al.  Efficient RANSAC for Point‐Cloud Shape Detection , 2007, Comput. Graph. Forum.

[3]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[4]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[5]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  Thomas A. Funkhouser,et al.  A benchmark for 3D mesh segmentation , 2009, ACM Trans. Graph..

[7]  Xianfeng Huang,et al.  A Methodology for Automated Segmentation and Reconstruction of Urban 3-D Buildings from ALS Point Clouds , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Tania Landes,et al.  AUTOMATIC SEGMENTATION OF BUILDING FACADES USING TERRESTRIAL LASER DATA , 2007 .

[9]  J. Shan,et al.  A global optimization approach to roof segmentation from airborne lidar point clouds , 2014 .

[10]  G. Sithole,et al.  Recognising structure in laser scanning point clouds , 2004 .

[11]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[12]  Peter V. Gehler,et al.  Efficient Facade Segmentation Using Auto-context , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[13]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[14]  G. Vosselman,et al.  SEGMENTATION OF UAV-BASED IMAGES INCORPORATING 3D POINT CLOUD INFORMATION , 2015 .

[15]  Ramesh C. Jain,et al.  Segmentation through Variable-Order Surface Fitting , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Kevin Jiang.  Introduction , 2013, Nature Medicine.

[17]  Mohammad Omidalizarandi,et al.  SEGMENTATION AND CLASSIFICATION OF POINT CLOUDS FROM DENSE AERIAL IMAGE MATCHING , 2013 .

[18]  R. Reulke,et al.  Remote Sensing and Spatial Information Sciences , 2005 .

[19]  Lei Tang,et al.  An Improved RANSAC for 3D Point Cloud Plane Segmentation Based on Normal Distribution Transformation Cells , 2017, Remote. Sens..

[20]  Irfan A. Essa,et al.  Efficient Hierarchical Graph-Based Segmentation of RGBD Videos , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Kristen Moore,et al.  3D Reconstruction, Segmentation and Classification of Corals from Aerial Images Final Report: CS 231A , 2014 .

[22]  F. Tarsha-Kurdi,et al.  Hough-Transform and Extended RANSAC Algorithms for Automatic Detection of 3D Building Roof Planes from Lidar Data , 2007 .

[23]  José Luis Lerma,et al.  Unsupervised robust planar segmentation of terrestrial laser scanner point clouds based on fuzzy clustering methods , 2008 .

[24]  Michela Bertolotto,et al.  Octree-based region growing for point cloud segmentation , 2015 .

[25]  Andrés Serna,et al.  Detection, segmentation and classification of 3D urban objects using mathematical morphology and supervised learning , 2014 .

[26]  C. Mallet,et al.  A structured regularization framework for spatially smoothing semantic labelings of 3D point clouds , 2017 .

[27]  Kenji Shimada,et al.  Surface mesh segmentation and smooth surface extraction through region growing , 2005, Comput. Aided Geom. Des..

[28]  P. Dorninger,et al.  3 D SEGMENTATION OF UNSTRUCTURED POINT CLOUDS FOR BUILDING MODELLING , 2007 .

[29]  Peter V. Gehler,et al.  Efficient 2D and 3D Facade Segmentation Using Auto-Context , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Abdul Nurunnabi,et al.  Robust Segmentation in Laser Scanning 3D Point Cloud Data , 2012, 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA).

[31]  Luc Van Gool,et al.  3D all the way: Semantic segmentation of urban scenes from start to end in 3D , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  T. Rabbani,et al.  EFFICIENT HOUGH TRANSFORM FOR AUTOMATIC DETECTION OF CYLINDERS IN POINT CLOUDS , 2005 .

[33]  B. Gorte.  SEGMENTATION OF TIN-STRUCTURED SURFACE MODELS , 2002 .