GIFT: A Real-Time and Scalable 3D Shape Search Engine

Projective analysis is an important solution for 3D shape retrieval, since human visual perceptions of 3D shapes rely on various 2D observations from different view points. Although multiple informative and discriminative views are utilized, most projection-based retrieval systems suffer from heavy computational cost, thus cannot satisfy the basic requirement of scalability for search engines. In this paper, we present a real-time 3D shape search engine based on the projective images of 3D shapes. The real-time property of our search engine results from the following aspects: (1) efficient projection and view feature extraction using GPU acceleration, (2) the first inverted file, referred as F-IF, is utilized to speed up the procedure of multi-view matching, (3) the second inverted file (S-IF), which captures a local distribution of 3D shapes in the feature manifold, is adopted for efficient context-based reranking. As a result, for each query the retrieval task can be finished within one second despite the necessary cost of IO overhead. We name the proposed 3D shape search engine, which combines GPU acceleration and Inverted File (Twice), as GIFT. Besides its high efficiency, GIFT also outperforms the state-of-the-art methods significantly in retrieval accuracy on various shape benchmarks and competitions.

[1]  Meng Wang,et al.  3D deep shape descriptor , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Afzal Godil,et al.  CM-BOF: visual similarity-based 3D shape retrieval using Clock Matching and Bag-of-Features , 2013, Machine Vision and Applications.

[4]  Ioannis Pratikakis,et al.  PANORAMA: A 3D Shape Descriptor Based on Panoramic Views for Unsupervised 3D Object Retrieval , 2010, International Journal of Computer Vision.

[5]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[6]  Hamid Laga,et al.  Compact Vectors of Locally Aggregated Tensors for 3D Shape Retrieval , 2013, 3DOR@Eurographics.

[7]  Bin Fang,et al.  A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries , 2015, Comput. Vis. Image Underst..

[8]  Horst Bischof,et al.  Diffusion Processes for Retrieval Revisited , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Olivier Colot,et al.  A New 3D-Matching Method of Nonrigid and Partially Similar Models Using Curve Analysis , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Iasonas Kokkinos,et al.  Scale-invariant heat kernel signatures for non-rigid shape recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[12]  Longin Jan Latecki,et al.  3D Shape Matching via Two Layer Coding , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Daniela Giorgi,et al.  SHape REtrieval Contest 2007: Watertight Models Track , 2007 .

[14]  Zhi-Hua Zhou,et al.  Semi-Supervised Regression with Co-Training , 2005, IJCAI.

[15]  Bo Li,et al.  3D model retrieval using hybrid features and class information , 2013, Multimedia Tools and Applications.

[16]  Qi Tian,et al.  Query-adaptive late fusion for image search and person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yasuo Kuniyoshi,et al.  Elastic Net Constraints for Shape Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[18]  Alexander M. Bronstein,et al.  Supervised learning of bag‐of‐features shape descriptors using sparse coding , 2014, Comput. Graph. Forum.

[19]  Maks Ovsjanikov,et al.  Persistence-Based Structural Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[21]  Ming Yang,et al.  Query Specific Rank Fusion for Image Retrieval , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Zhichao Zhou,et al.  DeepPano: Deep Panoramic Representation for 3-D Shape Recognition , 2015, IEEE Signal Processing Letters.

[23]  Song Wu,et al.  3 D ShapeNets : A Deep Representation for Volumetric Shape Modeling , 2015 .

[24]  Longin Jan Latecki,et al.  Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval , 2009, CVPR.

[25]  Alexander M. Bronstein,et al.  Scalability of Non-Rigid 3D Shape Retrieval , 2015, 3DOR@Eurographics.

[26]  Daniel Cremers,et al.  Dense Non-rigid Shape Correspondence Using Random Forests , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Daniel Cohen-Or,et al.  Projective analysis for 3D shape segmentation , 2013, ACM Trans. Graph..

[28]  Edward K. Wong,et al.  Deepshape: Deep learned shape descriptor for 3D shape matching and retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Haibin Ling,et al.  Shape Classification Using the Inner-Distance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Frank Nielsen,et al.  Shape Retrieval Using Hierarchical Total Bregman Soft Clustering , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Longin Jan Latecki,et al.  Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[33]  C. Schmid,et al.  On the burstiness of visual elements , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Qi Tian,et al.  Multimedia search reranking: A literature survey , 2014, CSUR.

[35]  Ali Shokoufandeh,et al.  Retrieving articulated 3-D models using medial surfaces , 2008, Machine Vision and Applications.

[36]  Konrad Schindler,et al.  VocMatch: Efficient Multiview Correspondence for Structure from Motion , 2014, ECCV.

[37]  Dejan V. Vranic.  DESIRE: a composite 3D-shape descriptor , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[38]  SchreckTobias,et al.  A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries , 2015 .

[39]  Hamid Laga,et al.  Covariance Descriptors for 3D Shape Matching and Retrieval , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[41]  Iasonas Kokkinos,et al.  Intrinsic shape context descriptors for deformable shapes , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Masaki Aono,et al.  A large-scale Shape Benchmark for 3D object retrieval: Toyohashi shape benchmark , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[43]  Yasuo Kuniyoshi,et al.  Efficient Shape Matching using Vector Extrapolation , 2013, BMVC.

[44]  Kostas Daniilidis,et al.  Spherical Correlation of Visual Representations for 3D Model Retrieval , 2009, International Journal of Computer Vision.

[45]  Ioannis Pratikakis,et al.  Efficient 3D shape matching and retrieval using a concrete radialized spherical projection representation , 2007, Pattern Recognit..

[46]  Ioannis Pratikakis,et al.  3D Object Retrieval using an Efficient and Compact Hybrid Shape Descriptor , 2008, 3DOR@Eurographics.

[47]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).