Spatial Transformer for 3D Point Clouds.

Deep neural networks can efficiently process 3D point clouds. At each point convolution layer, local features can be learned from local neighborhoods of point clouds. These features are combined together for further processing to extract the semantic information encoded in the point cloud. Previous networks adopt all the same local neighborhoods at different layers, as they utilize the same metric on fixed input point coordinates to define neighborhoods. It is easy to implement but not necessarily optimal. Ideally local neighborhoods should be different at different layers so as to adapt to layer dynamics for efficient feature learning. One way to achieve this is to learn transformations of the input point cloud at each layer, and extract features from local neighborhoods defined on transformed coordinates. We propose a novel approach to learn different transformations of the input point cloud for different neighborhoods at each layer. We propose both linear and non-linear spatial transformers for point clouds. The proposed methods outperform the state-of-the-art methods in several other point cloud processing tasks (classification, segmentation and detection). Visualizations show that transformers can learn features more efficiently by dynamically altering neighborhoods according to the geometric and semantic information of 3D shapes regardless of intra-class variations.