A 3D Shape Retrieval Framework Supporting Multimodal Queries

This paper presents a unified framework for 3D shape retrieval. The method supports multimodal queries (2D images, sketches, 3D objects) by introducing a novel view-based approach able to handle the different types of multimedia data. More specifically, a set of 2D images (multi-views) are automatically generated from a 3D object, by taking views from uniformly distributed viewpoints. For each image, a set of 2D rotation-invariant shape descriptors is produced. The global shape similarity between two 3D models is achieved by applying a novel matching scheme, which effectively combines the information extracted from the multi-view representation. The experimental results prove that the proposed method demonstrates superior performance over other well-known state-of-the-art approaches.

[1]  Tim Weyrich,et al.  Post-processing of Scanned 3D Surface Data , 2004, PBG.

[2]  Dietmar Saupe,et al.  Description of 3D-shape using a complex function on the sphere , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[3]  Frank B. ter Haar,et al.  SHREC 2009 - Shape Retrieval Contest , 2009, 3DOR@Eurographics.

[4]  Dejan V. Vranic,et al.  3D model retrieval , 2004 .

[5]  Petros Daras,et al.  SHREC '11 Track: Generic Shape Retrieval , 2009, 3DOR@Eurographics.

[6]  Majid Ahmadi,et al.  Pattern recognition with moment invariants: A comparative study and new results , 1991, Pattern Recognit..

[7]  Nikolaos Canterakis,et al.  3D Zernike Moments and Zernike Affine Invariants for 3D Image Analysis and Recognition , 1999 .

[8]  Karthik Ramani,et al.  An Approach to Drawing-Like View Generation From 3D Models , 2005 .

[9]  Ayellet Tal,et al.  Mesh retrieval by components , 2006, GRAPP.

[10]  Paul H. Lewis,et al.  SCULPTEUR: Multimedia Retrieval for Museums , 2004, CIVR.

[11]  Mohamed Daoudi,et al.  3D models retrieval by using characteristic views , 2002, Object recognition supported by user interaction for service robots.

[12]  Michael G. Strintzis,et al.  Three-Dimensional Shape-Structure Comparison Method for Protein Classification , 2006, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[13]  Dejan V. VraniC.  An improvement of rotation invariant 3D-shape based on functions on concentric spheres , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[14]  Hans-Peter Kriegel,et al.  Effective similarity search on voxelized CAD objects , 2003, Eighth International Conference on Database Systems for Advanced Applications, 2003. (DASFAA 2003). Proceedings..

[15]  Bernard Chazelle,et al.  Matching 3D models with shape distributions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[16]  Ryutarou Ohbuchi,et al.  Shape-similarity search of three-dimensional models using parameterized statistics , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[17]  Michael G. Strintzis,et al.  Efficient 3-D model search and retrieval using generalized 3-D radon transforms , 2006, IEEE Transactions on Multimedia.

[18]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[19]  Berthold K. P. Horn.  Extended Gaussian images , 1984, Proceedings of the IEEE.

[20]  Daniel A. Keim,et al.  Content-Based 3D Object Retrieval , 2007, IEEE Computer Graphics and Applications.

[21]  Yi Liu,et al.  The Generalized Shape Distributions for Shape Matching and Analysis , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[22]  Petros Daras,et al.  SHREC'09 Track: Structural Shape Retrieval on Watertight Models , 2009, 3DOR@Eurographics.

[23]  Tony Tung,et al.  Augmented Reeb Graphs for Content-Based Retrieval of 3D Mesh Models (Figures 2 and 15) , 2004, SMI.

[24]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[25]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[26]  Guojun Lu,et al.  Shape-based image retrieval using generic Fourier descriptor , 2002, Signal Process. Image Commun..

[27]  Ayellet Tal,et al.  Hierarchical mesh decomposition using fuzzy clustering and cuts , 2003, ACM Trans. Graph..

[28]  BENJAMIN BUSTOS,et al.  Feature-based similarity search in 3D object databases , 2005, CSUR.

[29]  Ming Ouhyoung,et al.  A 3D Object Retrieval System Based on Multi-Resolution Reeb Graph , 2002 .

[30]  Karthik Ramani,et al.  Three-dimensional shape searching: state-of-the-art review and future trends , 2005, Comput. Aided Des..

[31]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[32]  Michael G. Strintzis,et al.  3D Model Search and Retrieval Using the Spherical Trace Transform , 2007, EURASIP J. Adv. Signal Process..

[33]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[34]  Noel E. O'Connor,et al.  SHREC’08 entry: Multi-view 3D retrieval using multi-scale contour representation , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[35]  Martial Hebert,et al.  Fully automatic registration of multiple 3D data sets , 2003, Image Vis. Comput..

[36]  Eric Wahl,et al.  Surflet-pair-relation histograms: a statistical 3D-shape representation for rapid classification , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[37]  Greg Turk,et al.  Image-driven simplification , 2000, TOGS.

[38]  Hans-Peter Kriegel,et al.  3D Shape Histograms for Similarity Search and Classification in Spatial Databases , 1999, SSD.

[39]  Ioannis Pratikakis,et al.  Efficient 3D shape matching and retrieval using a concrete radialized spherical projection representation , 2007, Pattern Recognit..

[40]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[41]  Karthik Ramani,et al.  Developing an engineering shape benchmark for CAD models , 2006, Comput. Aided Des..

[42]  Petros Daras,et al.  SHREC 2009 - Shape Retrieval Contest of Partial 3D Models | NIST , 2009 .

[43]  Chang-Hsing Lee,et al.  A new 3D model retrieval approach based on the elevation descriptor , 2007, Pattern Recognit..

[44]  Mohamed Daoudi,et al.  A Bayesian 3-D Search Engine Using Adaptive Views Clustering , 2007, IEEE Transactions on Multimedia.

[45]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[47]  Raveendran Paramesran,et al.  Image analysis by Krawtchouk moments , 2003, IEEE Trans. Image Process..

[48]  Dietmar Saupe,et al.  Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[49]  Anne Verroust-Blondet,et al.  Enhanced 2D/3D Approaches Based on Relevance Index for 3D-Shape Retrieval , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[50]  Petros Daras,et al.  3D Object Retrieval based on Resulting Fields , 2008 .

[51]  Marcin Novotni,et al.  3D zernike descriptors for content based shape retrieval , 2003, SM '03.

[52]  Remco C. Veltkamp,et al.  A Survey of Content Based 3D Shape Retrieval Methods , 2004, SMI.

[53]  Tony Tung,et al.  The Augmented Multiresolution Reeb Graph Approach for Content-based Retrieval of 3d Shapes , 2005, Int. J. Shape Model..

[54]  Bernd Gärtner,et al.  The smallest enclosing ball of balls: combinatorial structure and algorithms , 2004, Int. J. Comput. Geom. Appl..

[55]  Ryutarou Ohbuchi,et al.  Salient local visual features for shape-based 3D model retrieval , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[56]  Ryutarou Ohbuchi,et al.  Shape-similarity search of 3D models by using enhanced shape functions , 2003, Proceedings of Theory and Practice of Computer Graphics, 2003..

[57]  Petros Daras,et al.  SHREC 2009 - Generic Shape Retrieval Contest , 2009 .

[58]  Beatrice Gralton,et al.  Washington DC - USA , 2008 .

[59]  TalAyellet,et al.  Hierarchical mesh decomposition using fuzzy clustering and cuts , 2003 .

[60]  M. Teague.  Image analysis via the general theory of moments , 1980 .

[61]  Michael G. Strintzis,et al.  Combining Topological and Geometrical Features for Global and Partial 3-D Shape Retrieval , 2008, IEEE Transactions on Multimedia.

[62]  Neville Maxwell,et al.  Hong Kong 1999 , 2000 .

[63]  Bernard Chazelle,et al.  Shape distributions , 2002, TOGS.

[64]  Mohamed Daoudi,et al.  3D-Model search engine from photos , 2007, CIVR '07.

[65]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[66]  Alfonso Padilla-Vivanco,et al.  Digital image reconstruction using Zernike moments , 2004, SPIE Remote Sensing.

[67]  Katsushi Ikeuchi,et al.  The Complex EGI: A New Representation for 3-D Pose Determination , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[68]  Michael G. Strintzis,et al.  3D Protein Classification using Topological, Geometrical and Biological Information , 2007, 2007 IEEE International Conference on Image Processing.