Learning Hierarchical Semantic Segmentations of LIDAR Data

This paper investigates a method for semantic segmentation of small objects in terrestrial LIDAR scans in urban environments. The core research contribution is a hierarchical segmentation algorithm where potential merges between segments are prioritized by a learned affinity function and constrained to occur only if they achieve a significantly high object classification probability. This approach provides a way to integrate a learned shape-prior (the object classifier) into a search for the best semantic segmentation in a fast and practical algorithm. Experiments with LIDAR scans collected by Google Street View cars throughout ~100 city blocks of New York City show that the algorithm provides better segmentations and classifications than simple alternatives for cars, vans, traffic lights, and street lights.

[1]  Ioannis Stamos,et al.  Classification of Vehicle Parts in Unstructured 3D Point Clouds , 2014, 2014 2nd International Conference on 3D Vision.

[2]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Martial Hebert,et al.  3-D scene analysis via sequenced predictions over points and regions , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Ke Xie,et al.  A search-classify approach for cluttered indoor scene understanding , 2012, ACM Trans. Graph..

[5]  Svetlana Lazebnik,et al.  Finding Things: Image Parsing with Regions and Per-Exemplar Detectors , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Dieter Fox,et al.  RGB-(D) scene labeling: Features and algorithms , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Ariel Shamir,et al.  Seam Carving for Content-Aware Image Resizing , 2007, ACM Trans. Graph..

[8]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[9]  Vladimir G. Kim,et al.  Shape-based recognition of 3D point clouds in urban environments , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Jitendra Malik,et al.  Aligning 3D models to RGB-D images of cluttered scenes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Dieter Fox,et al.  Detection-based object labeling in 3D scenes , 2012, 2012 IEEE International Conference on Robotics and Automation.

[12]  Jianxiong Xiao,et al.  Sliding Shapes for 3D Object Detection in Depth Images , 2014, ECCV.

[13]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[14]  Jitendra Malik,et al.  Recognizing Objects in Range Data Using Regional Point Descriptors , 2004, ECCV.

[15]  Jitendra Malik,et al.  Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation , 2015, International Journal of Computer Vision.

[16]  H. Sebastian Seung,et al.  Learning to Agglomerate Superpixel Hierarchies , 2011, NIPS.

[17]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[18]  Sanja Fidler,et al.  Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Konrad Schindler,et al.  IMPLICIT SHAPE MODELS FOR OBJECT DETECTION IN 3D POINT CLOUDS , 2012, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[20]  George Vosselman,et al.  Recognizing basic structures from mobile laser scanning data for road inventory studies , 2011 .

[21]  Shi-Min Hu,et al.  Structure recovery by part assembly , 2012, ACM Trans. Graph..

[22]  Luc Van Gool,et al.  Hough Transform and 3D SURF for Robust Three Dimensional Classification , 2010, ECCV.

[23]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Hongbin Zha,et al.  Segmentation and classification of range image from an intelligent vehicle in urban environment , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Dmitri B. Chklovskii,et al.  Machine learning of hierarchical clustering to segment n-dimensional images , 2013, ArXiv.

[26]  Ben Taskar,et al.  Discriminative learning of Markov random fields for segmentation of 3D scan data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Andrés Serna,et al.  Detection, segmentation and classification of 3D urban objects using mathematical morphology and supervised learning , 2014 .

[28]  Bisheng Yang,et al.  Hierarchical extraction of urban objects from mobile laser scanning data , 2015 .

[29]  Ahmad Kamal Aijazi,et al.  Segmentation Based Classification of 3D Urban Point Clouds: A Super-Voxel Based Approach with Evaluation , 2013, Remote. Sens..

[30]  Jitendra Malik,et al.  Recognition using regions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Frank Moosmann,et al.  Unsupervised discovery of object classes in 3D outdoor scenarios , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[32]  Dieter Fox,et al.  Unsupervised feature learning for 3D scene labeling , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Dieter Fox,et al.  3D laser scan classification using web data and domain adaptation , 2009, Robotics: Science and Systems.