Det2Seg: A Two-Stage Approach for Road Object Segmentation from 3D Point Clouds

Object segmentation from 3D point clouds is an important topic in real-world applications. However, due to the sparsity and irregularity of point clouds, instance segmentation often suffers unsatisfying performance in various scenes such as autonomous driving. There are two main difficulties. First, in a wild scene, background noise often occupies a majority of the entire point set. Second, small-scale objects are often difficult to be recognized due to larger uncertainty. In this paper, we propose Det2Seg, a two-stage approach which alleviates the above issues towards more accurate instance segmentation of road-objects. In the first stage, we adopt Pointpillars [1] to detect the regions-of- interest that can localize and classify objects in a coarse level; in the second stage, we extract points from the detected regions into pillars, encode them into a new data format and feed it into a 2D convolutional neural network to perform fine-grained, domain-specific instance segmentation. We evaluate our approach on raw LiDAR (Light Detection And Ranging) data from the KITTI dataset [2]. The experimental results show that our approach largely outperforms the prior researches. In particular, our approach stands out for its significant ability on recognizing and segmenting small-scale objects, i.e., an improvement of over 20% in terms of Intersection over Union (IoU), beyond state-of- the-arts, is obtained for the cyclist class.