A semantic tree-based approach for sketch-based 3D model retrieval

Sketch-based 3D model retrieval is to retrieve 3D models given a user's hand-drawn sketch. Due to the big semantic gap between rough sketch representation and accurate 3D model coordinates, sketch-based 3D model retrieval (SBR) is one of the most challenging research topics in the field of 3D model retrieval. To bridge the semantic gap, a novel semantic tree-based SBR algorithm is proposed in this paper. Given a 2D sketch query and a collection of 3D models, a 3D semantic tree is built up first based on the ontology structure of WordNet. Every leaf node in the tree contains a set of 3D models assigned to this class according to their semantic classification/label information. Then, sketch components of the 2D query sketch are identified by sketch segmentation and annotation. Finally, by measuring the semantic relatedness between the sketch components' annotations and tree nodes in the 3D semantic tree, the similarities between the 2D sketch and 3D models are computed to find out the most relevant 3D models. Experimental results demonstrate the effectiveness and promising potentials of our approach on sketch-based 3D model retrieval.

[1]  Cordelia Schmid,et al.  Semantic Hierarchies for Visual Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Rynson W. H. Lau,et al.  Data-driven segmentation and labeling of freehand sketches , 2014, ACM Trans. Graph..

[3]  Bo Li,et al.  3D model retrieval using hybrid features and class information , 2013, Multimedia Tools and Applications.

[4]  Stéphane Herbin,et al.  Semantic hierarchies for image annotation: A survey , 2012, Pattern Recognit..

[5]  Bo Li,et al.  A comparison of methods for sketch-based 3D shape retrieval , 2014, Comput. Vis. Image Underst..

[6]  Ted Pedersen,et al.  WordNet::Similarity - Measuring the Relatedness of Concepts , 2004, NAACL.

[7]  Ted Pedersen,et al.  Extended Gloss Overlaps as a Measure of Semantic Relatedness , 2003, IJCAI.

[8]  Bin Fang,et al.  A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries , 2015, Comput. Vis. Image Underst..

[9]  Ted Pedersen,et al.  Measures of semantic similarity and relatedness in the biomedical domain , 2007, J. Biomed. Informatics.

[10]  Siddharth Patwardhan,et al.  Incorporating Dictionary and Corpus Information into a Context Vector Measure of Semantic Relatednes , 2003 .

[11]  Bo Li,et al.  Sketch-Based 3D Model Retrieval by Viewpoint Entropy-Based Adaptive View Clustering , 2013, 3DOR@Eurographics.

[12]  Marc Alexa,et al.  SHREC'12 Track: Sketch-Based 3D Shape Retrieval , 2012, 3DOR@Eurographics.

[13]  Sanyuan Zhang,et al.  An Overview of Semantics Processing in Content-Based 3D Model Retrieval , 2009, 2009 International Conference on Artificial Intelligence and Computational Intelligence.

[14]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[15]  Arjan Kuijper,et al.  Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours , 2010, ACM Multimedia.

[16]  Ted Pedersen,et al.  Using Measures of Semantic Relatedness for Word Sense Disambiguation , 2003, CICLing.

[17]  Christiane Fellbaum,et al.  Combining Local Context and Wordnet Similarity for Word Sense Identification , 1998 .

[18]  Sheng-sheng Wang,et al.  An Ontology and SWRL Based 3D Model Retrieval System , 2008, AIRS.

[19]  Bo Li,et al.  View Context: A 3D Model Feature for Retrieval , 2010, MMM.

[20]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[21]  Bo Li,et al.  Extended Large Scale Sketch-Based 3D Shape Retrieval , 2014, 3DOR@Eurographics.

[22]  Christiane Fellbaum,et al.  Lexical Chains as Representations of Context for the Detection and Correction of Malapropisms , 1998 .

[23]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[24]  Michael E. Lesk,et al.  Automatic sense disambiguation using machine readable dictionaries: how to tell a pine cone from an ice cream cone , 1986, SIGDOC '86.

[25]  Bo Li,et al.  Sketch-based 3D model retrieval by incorporating 2D-3D alignment , 2012, Multimedia Tools and Applications.

[26]  Ted Pedersen,et al.  An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet , 2002, CICLing.

[27]  Clement T. Yu,et al.  Using semantic contents and WordNet in image retrieval , 1997, SIGIR '97.

[28]  Bo Li,et al.  SHREC'13 Track: Large Scale Sketch-Based 3D Shape Retrieval , 2013, 3DOR@Eurographics.