Shape-based recognition of 3D point clouds in urban environments

This paper investigates the design of a system for recognizing objects in 3D point clouds of urban environments. The system is decomposed into four steps: locating, segmenting, characterizing, and classifying clusters of 3D points. Specifically, we first cluster nearby points to form a set of potential object locations (with hierarchical clustering). Then, we segment points near those locations into foreground and background sets (with a graph-cut algorithm). Next, we build a feature vector for each point cluster (based on both its shape and its context). Finally, we label the feature vectors using a classifier trained on a set of manually labeled objects. The paper presents several alternative methods for each step. We quantitatively evaluate the system and tradeoffs of different alternatives in a truthed part of a scan of Ottawa that contains approximately 100 million points and 1000 objects of interest. Then, we use this truth data as a training set to recognize objects amidst approximately 1 billion points of the remainder of the Ottawa scan.

[1]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[2]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Jitendra Malik,et al.  Shape Context: A New Descriptor for Shape Matching and Object Recognition , 2000, NIPS.

[5]  Suya You,et al.  Urban Site Modeling from LiDAR , 2003, ICCSA.

[6]  Martial Hebert,et al.  Robust extraction of multiple structures from non-uniformly sampled data , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[7]  Christian Früh,et al.  Data Processing Algorithms for Generating Textured 3D Building Facade Meshes from Laser Scans and Camera Images , 2005, International Journal of Computer Vision.

[8]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[9]  Martial Hebert,et al.  Scale selection for classification of point-sampled 3D surfaces , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[10]  Ben Taskar,et al.  Discriminative learning of Markov random fields for segmentation of 3D scan data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Wolfram Burgard,et al.  Autonomous Terrain Mapping and Classification Using Hidden Markov Models , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[12]  Josiane Zerubia,et al.  Building Outline Extraction from Digital Elevation Models Using Marked Point Processes , 2007, International Journal of Computer Vision.

[13]  Martial Hebert,et al.  Rapid object indexing using locality sensitive hashing and joint 3D-signature space estimation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Martial Hebert,et al.  Natural terrain classification using three‐dimensional ladar data for ground robot mobility , 2006, J. Field Robotics.

[15]  George Vosselman,et al.  Segmentation of point clouds using smoothness constraints , 2006 .

[16]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[17]  Rakesh Kumar,et al.  Rapid and scalable 3D object recognition using LIDAR data , 2006, SPIE Defense + Commercial Sensing.

[18]  T. Rabbani,et al.  SEGMENTATION OF POINT CLOUDS USING SMOOTHNESS CONSTRAINT , 2006 .

[19]  Norbert Haala,et al.  Improving the Realism of Existing 3D City Models , 2006, 3D-GIS.

[20]  Jie Chen,et al.  Architectural Modeling from Sparsely Scanned Range Data , 2008, International Journal of Computer Vision.

[21]  Marc Pierrot Deseilligny,et al.  3D Building Reconstruction with Parametric Roof Superstructures , 2007, 2007 IEEE International Conference on Image Processing.

[22]  Rachida Amjoun,et al.  Conditional Random Field for 3D Point Clouds with Adaptive Data Reduction , 2007, CW 2007.

[23]  David Suter,et al.  Conditional Random Field for 3D Point Clouds with Adaptive Data Reduction , 2007, 2007 International Conference on Cyberworlds (CW'07).

[24]  Hui Xu,et al.  Knowledge and heuristic-based modeling of laser-scanned trees , 2007, TOGS.

[25]  Josiane Zerubia,et al.  Building reconstruction from a single DEM , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  B. Koch,et al.  A Lidar Point Cloud Based Procedure for Vertical Canopy Structure Analysis And 3D Single Tree Modelling in Forest , 2008, Sensors.

[27]  Juha Hyyppä,et al.  Retrieval Algorithms for Road Surface Modelling Using Laser-Based Mobile Mapping , 2008, Sensors.

[28]  Norbert Pfeifer,et al.  A Comprehensive Automated 3D Approach for Building Extraction, Reconstruction, and Regularization from Airborne Laser Scanning Point Clouds , 2008, Sensors.

[29]  A. Zakhor,et al.  Fast Surface Reconstruction and Segmentation with Ground-Based and Airborne LIDAR Range Data , 2009 .