PTINet: Converting 3D Points to 2D Images with Deconvolution for Point Cloud Classification

3D point cloud classification is a very important task for many applications such as AR/VR, human-computer interaction, environment modeling, remote sensing, etc. Over the past decade, lots of research have been conducted and great progress been made in this field. However, point cloud classification is still a very challenging problem due to the irregularity of point cloud data, which makes the most popular deep neural network difficult to be applied. In order to get a regular 2D representation for point cloud data, some researchers projected point cloud data to 2D images by following some predefined rules, and utilized convolution for further processing. In this paper, we introduce a new deep network to find proper regular representations. The basic observation is that each point is considered as a 1x1 image. Therefore, deconvolution can be applied to map 3D points to 2D images. We named this network PTINet (Point to Image Network). Instead of predefined mapping rules, PTINet has the ability to find better mapping by learning, which can preserve 3D shape information as much as possible. The experiments conducted on the ModelNet dataset demonstrate competitive results of the proposed method.

[1]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[2]  Leonidas J. Guibas,et al.  FPNN: Field Probing Neural Networks for 3D Data , 2016, NIPS.

[3]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[5]  Karthik Ramani,et al.  Deep Learning 3D Shape Surfaces Using Geometry Images , 2016, ECCV.

[6]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[7]  Xu Xu,et al.  Beam search for learning a deep Convolutional Neural Network of 3D shapes , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[8]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  David Silver,et al.  Learning values across many orders of magnitude , 2016, NIPS.

[10]  Zhichao Zhou,et al.  DeepPano: Deep Panoramic Representation for 3-D Shape Recognition , 2015, IEEE Signal Processing Letters.

[11]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  David Moloney,et al.  3D Object Recognition Based on Volumetric Representation Using Convolutional Neural Networks , 2016, AMDO.

[13]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[14]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Subhransu Maji,et al.  3D Shape Segmentation with Projective Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).