Combining color and shape descriptors for 3D model retrieval

Abstract Nowadays many three dimensional models feature color information together with the shape description. However current content-based retrieval schemes for 3D models are based on shape information only and ignore color clues. The significance of shape vs. color clues for 3D model retrieval is instead a fundamental issue still almost unexplored at this time. A possible approach is to extend shape-based 3D model retrieval methods of proven effectiveness in order to include color. This work follows such rationale and introduces an extended version of the spin-image descriptor that can account also for color data. The comparison of color descriptors is performed using a novel scheme that allows to recognize as similar also objects with different colors but with the same color distribution over the shape. Shape and color similarity are finally combined together by an algorithm based on fuzzy logic. Experimental results show how the joint use of color and shape data allows to obtain better results than each of the two types of information alone. Comparisons with state-of-the-art content-based retrieval methods for 3D models also show how the proposed scheme outperforms standard solutions on object classes with meaningful color information.

[1]  Alberto Del Bimbo,et al.  Content-based retrieval of 3D models , 2006, TOMCCAP.

[2]  Guido M. Cortelazzo,et al.  3D registration by textured spin-images , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[3]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[4]  Nicola Orio,et al.  Retrieval of Colored 3D Models , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[5]  José María Carazo,et al.  Spin Images and Neural Networks for Efficient Content-Based Retrieval in 3D Object Databases , 2002, CIVR.

[6]  Anne Verroust-Blondet,et al.  A New Descriptor for 2D Depth Image Indexing and 3D Model Retrieval , 2007, 2007 IEEE International Conference on Image Processing.

[7]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[8]  Alberto Del Bimbo,et al.  Content-Based Retrieval of 3-D Objects Using Spin Image Signatures , 2007, IEEE Transactions on Multimedia.

[9]  Michael G. Strintzis,et al.  3D object retrieval using the 3D shape impact descriptor , 2009, Pattern Recognit..

[10]  Ryutarou Ohbuchi,et al.  SHREC’08 entry: Local volumetric features for 3D model retrieval , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[11]  Remco C. Veltkamp,et al.  A survey of content based 3D shape retrieval methods , 2004, Proceedings Shape Modeling Applications, 2004..

[12]  M. T. Suzuki,et al.  A Web-based retrieval system for 3D polygonal models , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[13]  Jun Qin,et al.  Content based 3D model retrieval: A survey , 2008, 2008 International Workshop on Content-Based Multimedia Indexing.

[14]  Yue Gao,et al.  View-based 3D model retrieval with probabilistic graph model , 2010, Neurocomputing.

[15]  Yong-Jin Liu,et al.  3D model retrieval based on color + geometry signatures , 2011, The Visual Computer.

[16]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[17]  Yue Gao,et al.  3D model comparison using spatial structure circular descriptor , 2010, Pattern Recognit..

[18]  Konstantinos N. Plataniotis,et al.  Aggregation of color and shape features for hybrid query generation in content based visual information retrieval , 2005, Signal Process..

[19]  Petros Daras,et al.  A Compact Multi-view Descriptor for 3D Object Retrieval , 2009, 2009 Seventh International Workshop on Content-Based Multimedia Indexing.

[20]  Ashfaq A. Khokhar,et al.  Quantized CIELab* space and encoded spatial structure for scalable indexing of large color image archives , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[21]  Marc Rioux,et al.  Description of shape information for 2-D and 3-D objects , 2000, Signal Process. Image Commun..

[22]  Ioannis Pratikakis,et al.  SHREC’08 entry: 2D/3D hybrid , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[23]  Martial Hebert,et al.  Control of Polygonal Mesh Resolution for 3-D Computer Vision , 1998, Graph. Model. Image Process..

[24]  Maks Ovsjanikov,et al.  Article 1 A. M. Bronstein Shape Google: Geometric Words and Expressions for (20 pages) M. M. Bronstein Invariant Shape Retrieval , 2011 .

[25]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[26]  Remco C. Veltkamp,et al.  A Measure for Evaluating Retrieval Techniques based on Partially Ordered Ground Truth Lists , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[27]  Kostas Daniilidis,et al.  Spherical Correlation of Visual Representations for 3D Model Retrieval , 2009, International Journal of Computer Vision.

[28]  Ghassan Hamarneh,et al.  A Survey on Shape Correspondence , 2011, Comput. Graph. Forum.

[29]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[30]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[31]  Andrew E. Johnson,et al.  Spin-Images: A Representation for 3-D Surface Matching , 1997 .

[32]  Frank B. ter Haar,et al.  SHape REtrieval Contest (SHREC) 2008 , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[33]  Akihiro Yamamoto,et al.  SHREC’08 entry: Semi-supervised learning for semantic 3D model retrieval , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[34]  Kunihiro Chihara,et al.  Discriminative Spherical Wavelet Features for Content-Based 3D Model Retrieval , 2007, Int. J. Shape Model..

[35]  Petros Daras,et al.  Investigating the Effects of Multiple Factors Towards More Accurate 3-D Object Retrieval , 2012, IEEE Transactions on Multimedia.

[36]  Andrew E. Johnson,et al.  Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[38]  Ryutarou Ohbuchi,et al.  SHREC'12 Track: Generic 3D Shape Retrieval , 2012, 3DOR@Eurographics.

[39]  Noel E. O'Connor,et al.  SHREC’08 entry: Multi-view 3D retrieval using multi-scale contour representation , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[40]  Zhiyong Huang,et al.  Combining Shape and Color for Retrieval of 3D Models , 2009, 2009 Fifth International Joint Conference on INC, IMS and IDC.

[41]  C.-H. Lee,et al.  3D object retrieval system based on grid D2 , 2005 .

[42]  Yi Liu,et al.  The Generalized Shape Distributions for Shape Matching and Analysis , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).