Multi-layers CNNs for 3D Model Retrieval

Due to the rapid development of 3D capturing scanners and better visual process techniques, there is a huge increase of 3D models being uploaded and captured by users. 3D model retrieval has become a hot topic in computer vision. State-of-the-art methods leverage CNNs to solve this problem. But existing CNN architectures and approaches are unable to fully exploit the information of 3D representations. In order to improve the performance of 3D object retrieval algorithms, we proposed a multi-layers CNNs (MLCNN) structure for 3D model representation. First, we combine the 12 rendered views of a 3D object into one representative view, which becomes the actual input. Second, in order to save the global and local information for each 3D model, we aggregate every convolutional layer’s feature into a multi-layers descriptor after a simple PCA compression. Finally, the Euclidean metric is leveraged to compute the similarity between two different 3D models to complete the retrieval problem. The final comparing experiments and corresponding experimental results demonstrate the superiority of our approach.

[1]  Huanbo Luan,et al.  Discrete Collaborative Filtering , 2016, SIGIR.

[2]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Rongrong Ji,et al.  Learning High-Level Feature by Deep Belief Networks for 3-D Model Retrieval and Recognition , 2014, IEEE Transactions on Multimedia.

[4]  Yue Gao,et al.  Multi-Modal Clique-Graph Matching for View-Based 3D Model Retrieval , 2016, IEEE Transactions on Image Processing.

[5]  Meng Wang,et al.  Harvesting visual concepts for image search with complex queries , 2012, ACM Multimedia.

[6]  Meng Wang,et al.  Multimedia answering: enriching text QA with media information , 2011, SIGIR.

[7]  Subhransu Maji,et al.  3D Shape Segmentation with Projective Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Qiong Liu,et al.  A Survey of Recent View-based 3D Model Retrieval Methods , 2012, ArXiv.

[9]  Dietmar Saupe,et al.  3D Model Retrieval with Spherical Harmonics and Moments , 2001, DAGM-Symposium.

[10]  Yang Yang,et al.  Robust (Semi) Nonnegative Graph Embedding , 2014, IEEE Transactions on Image Processing.

[11]  Tat-Seng Chua,et al.  Online Collaborative Learning for Open-Vocabulary Visual Classifiers , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Berk Gökberk,et al.  3D shape-based face recognition using automatically registered facial surfaces , 2004, ICPR 2004.

[13]  David Moloney,et al.  3D Object Recognition Based on Volumetric Representation Using Convolutional Neural Networks , 2016, AMDO.

[14]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Edward K. Wong,et al.  DeepShape: Deep-Learned Shape Descriptor for 3D Shape Retrieval , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[17]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[18]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Yue Gao,et al.  Camera Constraint-Free View-Based 3-D Object Retrieval , 2012, IEEE Transactions on Image Processing.

[20]  Remco C. Veltkamp,et al.  Polyhedral Model Retrieval Using Weighted Point Sets , 2003, Int. J. Image Graph..

[21]  Tat-Seng Chua,et al.  Learning from Collective Intelligence , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[22]  Songfan Yang,et al.  Multi-scale Recognition with DAG-CNNs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Yue Gao,et al.  View-Based 3D Object Retrieval: Challenges and Approaches , 2014, IEEE MultiMedia.

[24]  Yue Gao,et al.  Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval , 2013, ACM Multimedia.

[25]  Meng Wang,et al.  Learning Visual Semantic Relationships for Efficient Visual Retrieval , 2015, IEEE Transactions on Big Data.

[26]  Yuting Su,et al.  Graph-based characteristic view set extraction and matching for 3D model retrieval , 2015, Inf. Sci..

[27]  Meng Wang,et al.  Oracle in Image Search: A Content-Based Approach to Performance Prediction , 2012, TOIS.

[28]  Mohamed Daoudi,et al.  A Bayesian 3-D Search Engine Using Adaptive Views Clustering , 2007, IEEE Transactions on Multimedia.

[29]  Yoshua Bengio,et al.  Maxout Networks , 2013, ICML.

[30]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[31]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[32]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[33]  Yu-Ting Su,et al.  View-Based 3-D Model Retrieval: A Benchmark , 2018, IEEE Transactions on Cybernetics.

[34]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[35]  Yoshua Bengio,et al.  Object Recognition with Gradient-Based Learning , 1999, Shape, Contour and Grouping in Computer Vision.

[36]  Jialie Shen,et al.  Forbidden City Great Wall Old SummerPalace Temple of Heaven Tiananmen Square Avenue of Stars Disneyland Resort Peninsular Hotel Tian Tan Budda Victoria Harbour Big Ben Buckingham Palace , 2016 .

[37]  Bülent Sankur,et al.  3D Model Retrieval Using Probability Density-Based Shape Descriptors , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[39]  Zhihan Lv,et al.  Style-sensitive 3D model retrieval through sketch-based queries , 2016, Journal of Intelligent & Fuzzy Systems.

[40]  Zhichao Zhou,et al.  DeepPano: Deep Panoramic Representation for 3-D Shape Recognition , 2015, IEEE Signal Processing Letters.

[41]  BENJAMIN BUSTOS,et al.  Feature-based similarity search in 3D object databases , 2005, CSUR.

[42]  Meng Wang,et al.  Multi-View Object Retrieval via Multi-Scale Topic Models , 2016, IEEE Transactions on Image Processing.