PANORAMA-Based Multi-Scale and Multi-Channel CNN for 3D Model Retrieval

In this paper, a novel method for the retrieval of 3D models is proposed. Firstly, the 2D panoramic view is extracted from each 3D model as representation. Secondly, a novel Multi-Scale and Multi-Channel CNN (MSMC-NN) is proposed to automatically generate the feature of each 3D model. Here, the Multi-Scale CNN can effectively save the local and global information of 3D model from panorama views. Finally, the features extracted from the full connect layer are used to handle the retrieval problem. In experiment section, the popular standard datasets ModelNet-10 and ModelNet-40 were used to demonstrate the performance of the proposed method. Meanwhile, some classic 3D model retrieval methods were leveraged as comparison methods in this paper. The corresponding experiments also demonstrate the superiority of our approach.

[1]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Uwe Stilla,et al.  Detection of fallen trees in ALS point clouds of a temperate forest by combining point/primitive-level shape descriptors , 2014 .

[3]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Thomas Brox,et al.  Orientation-boosted Voxel Nets for 3D Object Recognition , 2016, BMVC.

[5]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[7]  Fritz Albregtsen,et al.  Fast and exact computation of Cartesian geometric moments using discrete Green's theorem , 1996, Pattern Recognit..

[8]  Yuting Su,et al.  Graph-based characteristic view set extraction and matching for 3D model retrieval , 2015, Inf. Sci..

[9]  Ioannis Pratikakis,et al.  PANORAMA: A 3D Shape Descriptor Based on Panoramic Views for Unsupervised 3D Object Retrieval , 2010, International Journal of Computer Vision.

[10]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  An-An Liu,et al.  3D Object Retrieval Based on Multi-View Latent Variable Model , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Zhang Xiong,et al.  A 3D model recognition mechanism based on deep Boltzmann machines , 2015, Neurocomputing.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Zhang Xiong,et al.  3D Object retrieval based on viewpoint segmentation , 2015, Multimedia Systems.

[15]  Yue Gao,et al.  Continuous Probability Distribution Prediction of Image Emotions via Multitask Shared Sparse Regression , 2017, IEEE Transactions on Multimedia.

[16]  Ke Lu,et al.  3D model retrieval and classification by semi-supervised learning with content-based similarity , 2014, Inf. Sci..

[17]  Longin Jan Latecki,et al.  GIFT: A Real-Time and Scalable 3D Shape Search Engine , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[19]  Ioannis Pratikakis,et al.  Exploiting the PANORAMA Representation for Convolutional Neural Network Classification and Retrieval , 2017, 3DOR@Eurographics.

[20]  Weizhi Nie,et al.  3D object retrieval based on sparse coding in weak supervision , 2016, J. Vis. Commun. Image Represent..

[21]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[22]  Zhichao Zhou,et al.  DeepPano: Deep Panoramic Representation for 3-D Shape Recognition , 2015, IEEE Signal Processing Letters.

[23]  Yue Gao,et al.  Multi-View 3D Object Retrieval With Deep Embedding Network , 2016, IEEE Transactions on Image Processing.

[24]  Hong Liu,et al.  View-based 3D object retrieval with discriminative views , 2017, Neurocomputing.

[25]  Yasuyuki Matsushita,et al.  RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints , 2016, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.