From 2D Silhouettes to 3D Object Retrieval: Contributions and Benchmarking

Abstract3D retrieval has recently emerged as an important boost for 2D search techniques. This is mainly due to its several complementary aspects, for instance, enriching views in 2D image datasets, overcoming occlusion and serving in many real-world applications such as photography, art, archeology, and geolocalization. In this paper, we introduce a complete "2D photography to 3D object" retrieval framework. Given a (collection of) picture(s) or sketch(es) of the same scene or object, the method allows us to retrieve the underlying similar objects in a database of 3D models. The contribution of our method includes (i) a generative approach for alignment able to find canonical views consistently through scenes/objects and (ii) the application of an efficient but effective matching method used for ranking. The results are reported through the Princeton Shape Benchmark and the Shrec benchmarking consortium evaluated/compared by a third party. In the two gallery sets, our framework achieves very encouraging performance and outperforms the other runs.

[1]  Bernd Gärtner,et al.  The smallest enclosing ball of balls: combinatorial structure and algorithms , 2004, Int. J. Comput. Geom. Appl..

[2]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[3]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[4]  Dietmar Saupe,et al.  3D Model Retrieval with Spherical Harmonics and Moments , 2001, DAGM-Symposium.

[5]  Daniela Giorgi,et al.  A Comparison Framework for 3D Object Classification Methods , 2006, MRCS.

[6]  Berthold K. P. Horn.  Extended Gaussian images , 1984, Proceedings of the IEEE.

[7]  Stefano Soatto,et al.  Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance , 2005, International Journal of Computer Vision.

[8]  Guofu Zhou,et al.  L , 2021, Edinburgh Medical and Surgical Journal.

[9]  Tony Tung,et al.  The Augmented Multiresolution Reeb Graph Approach for Content-based Retrieval of 3d Shapes , 2005, Int. J. Shape Model..

[10]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[11]  Mohamed Daoudi,et al.  A Bayesian 3-D Search Engine Using Adaptive Views Clustering , 2007, IEEE Transactions on Multimedia.

[12]  Dejan V. Vranic,et al.  3D model retrieval , 2004 .

[13]  Alberto Del Bimbo,et al.  Content-based retrieval of 3D models , 2006, TOMCCAP.

[14]  E. C. Leek,et al.  Effects of stimulus orientation on the identification of common polyoriented objects , 1998 .

[15]  Masayuki Nakajima,et al.  Spherical Wavelet Descriptors for Content-based 3D Model Retrieval , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[16]  Ioannis Pratikakis,et al.  SHREC’08 entry: 2D/3D hybrid , 2008, 2008 IEEE International Conference on Shape Modeling and Applications.

[17]  Matthew Goldstein,et al.  Kn -nearest Neighbor Classification , 1972, IEEE Trans. Inf. Theory.

[18]  Dietmar Saupe,et al.  Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[19]  Michael G. Strintzis,et al.  3D Model Search and Retrieval Using the Spherical Trace Transform , 2007, EURASIP J. Adv. Signal Process..

[20]  Silvia Biasotti,et al.  Sub-part correspondence by structural descriptors of 3D shapes , 2006, Comput. Aided Des..

[21]  Ryutarou Ohbuchi,et al.  Unsupervised learning from a corpus for shape-based 3D model retrieval , 2006, MIR '06.

[22]  Bernard Chazelle,et al.  Matching 3D models with shape distributions , 2001, Proceedings International Conference on Shape Modeling and Applications.

[23]  J. Tierny,et al.  3D Mesh Skeleton Extraction Using Topological and Geometrical Analyses , 2006 .

[24]  Dejan V. VraniC.  An improvement of rotation invariant 3D-shape based on functions on concentric spheres , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[25]  Marcin Novotni,et al.  3D zernike descriptors for content based shape retrieval , 2003, SM '03.

[26]  Remco C. Veltkamp,et al.  A Survey of Content Based 3D Shape Retrieval Methods , 2004, SMI.

[27]  Hans-Peter Kriegel,et al.  Nearest Neighbor Classification in 3D Protein Databases , 1999, ISMB.

[28]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[29]  Noel E. O'Connor,et al.  A multiscale representation method for nonrigid shapes with a single closed contour , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Daniel A. Keim,et al.  An experimental comparison of feature-based 3D retrieval methods , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[31]  Titus B. Zaharia,et al.  3D versus 2D/3D shape descriptors: a comparative study , 2004, IS&T/SPIE Electronic Imaging.

[32]  Anne Verroust-Blondet,et al.  Enhanced 2D/3D Approaches Based on Relevance Index for 3D-Shape Retrieval , 2006, IEEE International Conference on Shape Modeling and Applications 2006 (SMI'06).

[33]  Thomas A. Funkhouser,et al.  Partial matching of 3D shapes with priority-driven search , 2006, SGP '06.

[34]  Mohamed Daoudi,et al.  Invariant High Level Reeb Graphs of 3D Polygonal Meshes , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[35]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[36]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[37]  Anne Verroust-Blondet,et al.  A New Descriptor for 2D Depth Image Indexing and 3D Model Retrieval , 2007, 2007 IEEE International Conference on Image Processing.

[38]  Petros Daras,et al.  SHREC'09 Track: Structural Shape Retrieval on Watertight Models , 2009, 3DOR@Eurographics.

[39]  Katsushi Ikeuchi,et al.  Determining 3-D object pose using the complex extended Gaussian image , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Mohamed Daoudi,et al.  3D models retrieval by using characteristic views , 2002, Object recognition supported by user interaction for service robots.