Super-vector coding features extracted from both depth buffer and view-normal-angle images for part-based 3D shape retrieval

We have witnessed 3D shape models abundant in many application fields including 3D CAD/CAM, augmented/mixed reality (AR/MR), and entertainment. Creating 3D shape models from scratch is still very expensive. Efficient and accurate methods for shape retrieval is essential for 3D shape models to be reused. To retrieve similar 3D shape models, one must provide an arbitrary 3D shape as a query. Most of the research on 3D shape retrieval has been conducted with a “whole” shape as a query (aka whole-to-whole shape retrieval), while a “part” shape (aka part-to-whole shape retrieval) is more practically requested as a query especially by mechanical engineering with 3D CAD/CAM applications. A “part” shape is naturally constructed by a 3D range scanner as an input device. In this paper, we focus on the efficient method for part-to-whole shape retrieval where the “part” shape is assumed to be given by a 3D range scanner. Specifically, we propose a Super-Vector coding feature with SURF local features extracted from the View-Normal-Angle image, or the image synthesized by taking account of the angle between the view vector and the surface normal vector, together with the depth-buffered image, for part-to-whole shape retrieval. In addition, we propose a weighted whole-to-whole re-ranking method taking advantage of global information based on the result of part-to-whole shape retrieval. Through experiments we demonstrate that our proposed method outperforms the previous methods with or without re-ranking.

[1]  Thomas S. Huang,et al.  Image Classification Using Super-Vector Coding of Local Image Descriptors , 2010, ECCV.

[2]  Yosi Keller,et al.  Scale-Invariant Features for 3-D Mesh Models , 2012, IEEE Transactions on Image Processing.

[3]  Longin Jan Latecki,et al.  Locally constrained diffusion process on locally densified distance spaces with applications to shape retrieval , 2009, CVPR.

[4]  Subhransu Maji,et al.  Multi-view Convolutional Neural Networks for 3D Shape Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Jun Wang,et al.  From Low-Cost Depth Sensors to CAD: Cross-Domain 3D Shape Retrieval via Regression Tree Fields , 2014, ECCV.

[6]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[7]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[8]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[10]  Huarui Yin,et al.  Range Scans based 3D Shape Retrieval , 2015, 3DOR@Eurographics.

[11]  Michalis A. Savelonas,et al.  Fisher encoding of differential fast point feature histograms for partial 3D object retrieval , 2016, Pattern Recognit..

[12]  Anestis Koutsoudis,et al.  Partial matching of 3D cultural heritage objects using panoramic views , 2016, Multimedia Tools and Applications.

[13]  Bin Fang,et al.  A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries , 2015, Comput. Vis. Image Underst..

[14]  Tobias Schreck,et al.  SHREC'13 Track: Large-Scale Partial Shape Retrieval Using Simulated Range Images , 2013, 3DOR@Eurographics.

[15]  Ying Wu,et al.  Object retrieval and localization with spatially-constrained similarity measure and k-NN re-ranking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Sebastian Nowozin,et al.  Regression Tree Fields — An efficient, non-parametric approach to image labeling problems , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Bo Li,et al.  Sketch-based 3D model retrieval by incorporating 2D-3D alignment , 2012, Multimedia Tools and Applications.

[18]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[19]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[20]  Rita Cucchiara,et al.  GOLD: Gaussians of Local Descriptors for image representation , 2015, Comput. Vis. Image Underst..

[21]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[22]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[23]  Andrew Zisserman,et al.  Three things everyone should know to improve object retrieval , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Mohammed Bennamoun,et al.  Rotational Projection Statistics for 3D Local Surface Description and Object Recognition , 2013, International Journal of Computer Vision.

[26]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark (Figures 1 and 2) , 2004, Shape Modeling International Conference.

[27]  Jiaolong Yang,et al.  Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Ioannis Pratikakis,et al.  PANORAMA: A 3D Shape Descriptor Based on Panoramic Views for Unsupervised 3D Object Retrieval , 2010, International Journal of Computer Vision.

[29]  Ralph R. Martin,et al.  Partial Shape Queries for 3D Object Retrieval , 2016, 3DOR@Eurographics.

[30]  Ryutarou Ohbuchi,et al.  Randomized Sub-Volume Partitioning for Part-Based 3D Model Retrieval , 2015, 3DOR@Eurographics.

[31]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[32]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[33]  Songqiao Tao,et al.  Partial retrieval of CAD models based on local surface region decomposition , 2013, Comput. Aided Des..

[34]  Ryutarou Ohbuchi,et al.  Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[35]  Andrew Zisserman,et al.  The devil is in the details: an evaluation of recent feature encoding methods , 2011, BMVC.

[36]  Shin'ichi Satoh,et al.  Query-Adaptive Asymmetrical Dissimilarities for Visual Object Retrieval , 2013, 2013 IEEE International Conference on Computer Vision.

[37]  Bernhard Schölkopf,et al.  Ranking on Data Manifolds , 2003, NIPS.

[38]  Ryutarou Ohbuchi,et al.  Similarity metric learning for sketch-based 3D object retrieval , 2014, Multimedia Tools and Applications.

[39]  Shih-Fu Chang,et al.  3D shape retrieval using a single depth image from low-cost sensors , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[40]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[41]  Masaki Aono,et al.  Multi-Fourier spectra descriptor and augmentation with spectral clustering for 3D shape retrieval , 2009, The Visual Computer.

[42]  Kai Tang,et al.  Polynomial local shape descriptor on interest points for 3D part-in-whole matching , 2015, Comput. Aided Des..

[43]  Ryutarou Ohbuchi,et al.  SHREC'10 Track: Range Scan Retrieval , 2010, 3DOR@Eurographics.

[44]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..