View-Based 3-D Model Retrieval: A Benchmark

View-based 3-D model retrieval is one of the most important techniques in numerous applications of computer vision. While many methods have been proposed in recent years, to the best of our knowledge, there is no benchmark to evaluate the state-of-the-art methods. To tackle this problem, we systematically investigate and evaluate the related methods by: 1) proposing a clique graph-based method and 2) reimplementing six representative methods. Moreover, we concurrently evaluate both hand-crafted visual features and deep features on four popular datasets (NTU60, NTU216, PSB, and ETH) and one challenging real-world multiview model dataset (MV-RED) prepared by our group with various evaluation criteria to understand how these algorithms perform. By quantitatively analyzing the performances, we discover the graph matching-based method with deep features, especially the clique graph matching algorithm with convolutional neural networks features, can usually outperform the others. We further discuss the future research directions in this field.

[1]  Xiangyu Wang,et al.  3D Model Retrieval with Weighted Locality-constrained Group Sparse Coding , 2015, Neurocomputing.

[2]  Ryutarou Ohbuchi,et al.  Hashing Cross-Modal Manifold for Scalable Sketch-Based 3D Model Retrieval , 2014, 2014 2nd International Conference on 3D Vision.

[3]  Chun-Wei Tan,et al.  Accurate Iris Recognition at a Distance Using Stabilized Iris Encoding and Zernike Moments Phase Features , 2014, IEEE Transactions on Image Processing.

[4]  Zan Gao,et al.  Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition , 2015, Signal Process..

[5]  Xuelong Li,et al.  Spectral Multimodal Hashing and Its Application to Multimedia Retrieval , 2016, IEEE Transactions on Cybernetics.

[6]  Zhang Xiong,et al.  3D Object Retrieval With Multitopic Model Combining Relevance Feedback and LDA Model , 2015, IEEE Transactions on Image Processing.

[7]  Qi Tian,et al.  Image Search Reranking With Hierarchical Topic Awareness , 2015, IEEE Transactions on Cybernetics.

[8]  Zhiwen Yu,et al.  Hybrid Associative Retrieval of Three-Dimensional Models , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Dejan V. Vranic,et al.  3D model retrieval , 2004 .

[11]  Weizhi Nie,et al.  Clique-graph matching by preserving global & local structure , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Bo Li,et al.  3D model retrieval using hybrid features and class information , 2013, Multimedia Tools and Applications.

[13]  Yue Gao,et al.  Camera Constraint-Free View-Based 3-D Object Retrieval , 2012, IEEE Transactions on Image Processing.

[14]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[15]  Ryutarou Ohbuchi,et al.  Fusing Multiple Features for Shape-based 3D Model Retrieval , 2014, BMVC.

[16]  Bo Li,et al.  Sketch-Based 3D Model Retrieval by Viewpoint Entropy-Based Adaptive View Clustering , 2013, 3DOR@Eurographics.

[17]  Huarui Yin,et al.  Range Scans based 3D Shape Retrieval , 2015, 3DOR@Eurographics.

[18]  Yue Gao,et al.  Multi-Modal Clique-Graph Matching for View-Based 3D Model Retrieval , 2016, IEEE Transactions on Image Processing.

[19]  Hai Jin,et al.  Content-Based Visual Landmark Search via Multimodal Hypergraph Learning , 2015, IEEE Transactions on Cybernetics.

[20]  Yuan Yan Tang,et al.  High-Order Distance-Based Multiview Stochastic Learning in Image Classification , 2014, IEEE Transactions on Cybernetics.

[21]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[22]  Petros Daras,et al.  SHREC 2009 - Shape Retrieval Contest of Partial 3D Models | NIST , 2009 .

[23]  Ekta Walia,et al.  Variants of dense descriptors and Zernike moments as features for accurate shape-based image retrieval , 2014, Signal Image Video Process..

[24]  Chang-Hsing Lee,et al.  A new 3D model retrieval approach based on the elevation descriptor , 2007, Pattern Recognit..

[25]  Pritee Khanna,et al.  Computer-Aided Diagnosis of Malignant Mammograms using Zernike Moments and SVM , 2015, Journal of Digital Imaging.

[26]  Ioannis Pratikakis,et al.  3D object retrieval via range image queries in a bag-of-visual-words context , 2013, The Visual Computer.

[27]  Li Li,et al.  A Survey on Visual Content-Based Video Indexing and Retrieval , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[28]  Marc Alexa,et al.  SHREC'12 Track: Sketch-Based 3D Shape Retrieval , 2012, 3DOR@Eurographics.

[29]  Weizhi Nie,et al.  3D object retrieval based on sparse coding in weak supervision , 2016, J. Vis. Commun. Image Represent..

[30]  Sheng Tang,et al.  Sparse Ensemble Learning for Concept Detection , 2012, IEEE Transactions on Multimedia.

[31]  P. Danielsson Euclidean distance mapping , 1980 .

[32]  Fabio Roli,et al.  Neural shape codes for 3D model retrieval , 2015, Pattern Recognit. Lett..

[33]  Weizhi Nie,et al.  Effective 3D object detection based on detector and tracker , 2016, Neurocomputing.

[34]  Xiangfeng Luo,et al.  Topic Model for Graph Mining , 2015, IEEE Transactions on Cybernetics.

[35]  Weizhi Nie,et al.  Spatial Context Constrained Characteristic View Extraction for 3D Model Retrieval , 2015 .

[36]  Xuelong Li,et al.  Spectral Embedded Hashing for Scalable Image Retrieval , 2014, IEEE Transactions on Cybernetics.

[37]  Thierry Pun,et al.  Performance evaluation in content-based image retrieval: overview and proposals , 2001, Pattern Recognit. Lett..

[38]  Jin-Hui Zhu,et al.  Simultaneous Recognition and Modeling for Learning 3-D Object Models From Everyday Scenes , 2015, IEEE Transactions on Cybernetics.

[39]  Yue Gao,et al.  3D model retrieval using weighted bipartite graph matching , 2011, Signal Process. Image Commun..

[40]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Maks Ovsjanikov,et al.  SHREC’16 Track: 3D Sketch-Based 3D Shape Retrieval , 2016 .

[42]  Bo Li,et al.  Large-Scale 3D Shape Retrieval from ShapeNet Core55 , 2016, 3DOR@Eurographics.

[43]  Mohamed Daoudi,et al.  A Bayesian 3-D Search Engine Using Adaptive Views Clustering , 2007, IEEE Transactions on Multimedia.

[44]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Petros Daras,et al.  A 3D Shape Retrieval Framework Supporting Multimodal Queries , 2010, International Journal of Computer Vision.

[46]  Qi Tian,et al.  Less is More: Efficient 3-D Object Retrieval With Query View Selection , 2011, IEEE Transactions on Multimedia.

[47]  Xuelong Li,et al.  Event-Based Media Enrichment Using an Adaptive Probabilistic Hypergraph Model , 2015, IEEE Transactions on Cybernetics.

[48]  Yen-Wei Chen,et al.  High-Order Statistics of Weber Local Descriptors for Image Representation , 2015, IEEE Transactions on Cybernetics.

[49]  Ling Shao,et al.  Learning View-Model Joint Relevance for 3D Object Retrieval , 2015, IEEE Transactions on Image Processing.

[50]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Tat-Seng Chua,et al.  Learning from Collective Intelligence , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[52]  Fang Wang,et al.  Sketch-based 3D shape retrieval using Convolutional Neural Networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Heung-Kyu Lee,et al.  Fingerprint-PKI authentication using Zernike moments , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[54]  Weizhi Nie,et al.  Convolutional deep learning for 3D object retrieval , 2017, Multimedia Systems.

[55]  Paul L. Rosin,et al.  SHREC’16 Track: Partial Shape Queries for 3D Object Retrieval , 2016 .

[56]  Yue Gao,et al.  SHREC'15 Track: 3D Object Retrieval with Multimodal Views , 2015 .

[57]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[58]  Xindong Wu,et al.  3-D Object Retrieval With Hausdorff Distance Learning , 2014, IEEE Transactions on Industrial Electronics.

[59]  Qionghai Dai,et al.  View-based 3-D Object Retrieval , 2014 .

[60]  Yuting Su,et al.  Graph-based characteristic view set extraction and matching for 3D model retrieval , 2015, Inf. Sci..

[61]  Meng Wang,et al.  Oracle in Image Search: A Content-Based Approach to Performance Prediction , 2012, TOIS.

[62]  Yongdong Zhang,et al.  Visual stem mapping and Geometric Tense coding for Augmented Visual Vocabulary , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Yue Gao,et al.  3D model comparison using spatial structure circular descriptor , 2010, Pattern Recognit..

[64]  Wen Gao,et al.  Mining Compact Bag-of-Patterns for Low Bit Rate Mobile Visual Search , 2014, IEEE Transactions on Image Processing.

[65]  Céline Loscos,et al.  3D Model Retrieval , 2013 .

[66]  Yao Zhao,et al.  Cross-Modal Retrieval With CNN Visual Features: A New Baseline , 2017, IEEE Transactions on Cybernetics.

[67]  Alexander G. Hauptmann,et al.  Which Information Sources are More Effective and Reliable in Video Search , 2016, SIGIR.

[68]  Bo Li,et al.  SHREC'13 Track: Large Scale Sketch-Based 3D Shape Retrieval , 2013, 3DOR@Eurographics.

[69]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[70]  Ming Gao,et al.  BiRank: Towards Ranking on Bipartite Graphs , 2017, IEEE Transactions on Knowledge and Data Engineering.

[71]  Y. Rui,et al.  Learning to Rank Using User Clicks and Visual Features for Image Retrieval , 2015, IEEE Transactions on Cybernetics.

[72]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[73]  Ralph R. Martin,et al.  Partial Shape Queries for 3D Object Retrieval , 2016, 3DOR@Eurographics.