Learning Combinatorial Global and Local Features on 3D Point Clouds

As deep learning methods won the huge success in 2D area, they're striding forward into 3D computer vision community during the last few years, meanwhile as point cloud data becoming ubiquitous with rapid development of 3D sensors, deep learning implemented on 3D point cloud steps into different scenarios as 3D classification, segmentation, object detection and reconstruction. However, processing 3D point cloud data by deep learning is a non-trivial job because of its naturally irregular data format. Some predecessors overcame the key issue by designing a direct deep network, left the problem neglecting implicit local feature relations as an open question, though. Our proposal network, dual-channel multi-layer perceptron deep network, or DualNet, aiming to take advantage of local features and to learn from combinatorial global and local features, thus to achieve both stable order invariance and high accuracy for 3D object classification tasks. A couple of experiments will be conducted to verify the performance of our proposal DualNet by comparing with state-of-the-art methods on major 3D point cloud database.

[1]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[2]  Ralph R. Martin,et al.  Non-rigid 3D Shape Retrieval , 2015, 3DOR@Eurographics.

[3]  Iasonas Kokkinos,et al.  Scale-invariant heat kernel signatures for non-rigid shape recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Peng-Bo Zhang,et al.  A Novel AdaBoost Framework With Robust Threshold and Structural Optimization , 2018, IEEE Transactions on Cybernetics.

[5]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Leonidas J. Guibas,et al.  FPNN: Field Probing Neural Networks for 3D Data , 2016, NIPS.

[8]  Victor S. Lempitsky,et al.  Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Xianghua Xie,et al.  Graph Based Convolutional Neural Network , 2016, ArXiv.

[10]  Wolfram Burgard,et al.  Multimodal deep learning for robust RGB-D object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Baoquan Chen,et al.  PointCNN , 2018, NIPS 2018.

[12]  Iasonas Kokkinos,et al.  Intrinsic shape context descriptors for deformable shapes , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Dong Tian,et al.  Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[15]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[16]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Dong Tian,et al.  FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Sven Behnke,et al.  Fast Range Image Segmentation and Smoothing Using Approximate Surface Reconstruction and Region Growing , 2012, IAS.

[19]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[20]  Yang Liu,et al.  O-CNN , 2017, ACM Trans. Graph..

[21]  Gernot Riegler,et al.  OctNet: Learning Deep 3D Representations at High Resolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Xian-Bo Wang,et al.  Single and Simultaneous Fault Diagnosis With Application to a Multistage Gearbox: A Versatile Dual-ELM Network Approach , 2018, IEEE Transactions on Industrial Informatics.

[23]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[24]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[25]  A. Ben Hamza,et al.  Deep learning with geodesic moments for 3D shape classification , 2018, Pattern Recognit. Lett..

[26]  Leonidas J. Guibas,et al.  Volumetric and Multi-view CNNs for Object Classification on 3D Data , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Xavier Bresson,et al.  CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral Filters , 2017, IEEE Transactions on Signal Processing.

[28]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[29]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[31]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[32]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Nikos Komodakis,et al.  Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).