A comparison of methods for sketch-based 3D shape retrieval

Sketch-based 3D shape retrieval has become an important research topic in content-based 3D object retrieval. To foster this research area, two Shape Retrieval Contest (SHREC) tracks on this topic have been organized by us in 2012 and 2013 based on a small-scale and large-scale benchmarks, respectively. Six and five (nine in total) distinct sketch-based 3D shape retrieval methods have competed each other in these two contests, respectively. To measure and compare the performance of the top participating and other existing promising sketch-based 3D shape retrieval methods and solicit the state-of-the-art approaches, we perform a more comprehensive comparison of fifteen best (four top participating algorithms and eleven additional state-of-the-art methods) retrieval methods by completing the evaluation of each method on both benchmarks. The benchmarks, results, and evaluation tools for the two tracks are publicly available on our websites [1,2].

[1]  Daniel Cohen-Or,et al.  Sketch‐to‐Design: Context‐Based Part Assembly , 2012, Comput. Graph. Forum.

[2]  Bo Li,et al.  SHREC'13 Track: Large Scale Sketch-Based 3D Shape Retrieval , 2013, 3DOR@Eurographics.

[3]  Bo Li,et al.  Sketch-based 3D model retrieval by incorporating 2D-3D alignment , 2012, Multimedia Tools and Applications.

[4]  Manuel J. Fonseca,et al.  Sketch-based retrieval of drawings using spatial proximity , 2010, J. Vis. Lang. Comput..

[5]  Atilla Baskurt,et al.  Improving Zernike Moments Comparison for Optimal Similarity and Rotation Angle Retrieval , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Masaki Aono,et al.  Multi-Fourier spectra descriptor and augmentation with spectral clustering for 3D shape retrieval , 2009, The Visual Computer.

[7]  Guojun Lu,et al.  A Comparative Study of Fourier Descriptors for Shape Representation and Retrieval , 2002 .

[8]  Ilan Shimshoni,et al.  Multi-scale Curve Detection on Surfaces , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Marc Alexa,et al.  A descriptor for large scale image retrieval based on sketched feature lines , 2009, SBIM '09.

[10]  Ying He,et al.  Real-time computation of photic extremum lines (PELs) , 2010, The Visual Computer.

[11]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[12]  Bo Li,et al.  3D model retrieval using hybrid features and class information , 2013, Multimedia Tools and Applications.

[13]  Satoshi Kanai.  Content-based 3D mesh model retrieval from hand-written sketch , 2008 .

[14]  Tobias Schreck,et al.  Sketch-based 3D Model Retrieval using Keyshapes for Global and Local Representation , 2012, 3DOR@Eurographics.

[15]  Guojun Lu,et al.  Image Retrieval Using Modified Generic Fourier Descriptors , 2004, Computers and Their Applications.

[16]  Frédo Durand,et al.  Apparent ridges for line drawing , 2007, ACM Trans. Graph..

[17]  Shi-Min Hu,et al.  Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models , 2013, ACM Trans. Graph..

[18]  Bo Li,et al.  Sketch-Based 3D Model Retrieval by Viewpoint Entropy-Based Adaptive View Clustering , 2013, 3DOR@Eurographics.

[19]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[21]  Arjan Kuijper,et al.  View-based 3D model retrieval using compressive sensing based classification , 2011, 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA).

[22]  Bo Li,et al.  View Context: A 3D Model Feature for Retrieval , 2010, MMM.

[23]  Yong Jae Lee,et al.  ShadowDraw: real-time user guidance for freehand drawing , 2011, ACM Trans. Graph..

[24]  Szymon Rusinkiewicz,et al.  Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors , 2003, Symposium on Geometry Processing.

[25]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[26]  Marcin Novotni,et al.  3D zernike descriptors for content based shape retrieval , 2003, SM '03.

[27]  Marc Alexa,et al.  SHREC'12 Track: Sketch-Based 3D Shape Retrieval , 2012, 3DOR@Eurographics.

[28]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[29]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[30]  Jieqing Feng,et al.  Perceptual-saliency extremum lines for 3D shape illustration , 2010, The Visual Computer.

[31]  Ryutarou Ohbuchi,et al.  Scale-weighted dense bag of visual features for 3D model retrieval from a partial view 3D model , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[32]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[33]  Ryutarou Ohbuchi,et al.  Ranking on Cross-Domain Manifold for Sketch-Based 3D Model Retrieval , 2013, 2013 International Conference on Cyberworlds.

[34]  Yu Chen,et al.  Decomposition of Complex Line Drawings with Hidden Lines for 3D Planar-Faced Manifold Object Reconstruction , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Ying He,et al.  Real-Time Shape Illustration Using Laplacian Lines , 2011, IEEE Transactions on Visualization and Computer Graphics.

[37]  Pavel Zezula,et al.  Similarity Search: The Metric Space Approach (Advances in Database Systems) , 2005 .

[38]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. , 1980 .

[39]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[40]  Hanan Samet,et al.  Foundations of multidimensional and metric data structures , 2006, Morgan Kaufmann series in data management systems.

[41]  Patrick Bonin,et al.  A new set of 299 pictures for psycholinguistic studies: French norms for name agreement, image agreement, conceptual familiarity, visual complexity, image variability, age of acquisition, and naming latencies , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[42]  Benjamin Bustos,et al.  An Improved Histogram of Edge Local Orientations for Sketch-Based Image Retrieval , 2010, DAGM-Symposium.

[43]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Arjan Kuijper,et al.  Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours , 2010, ACM Multimedia.

[45]  Angel Fernandez,et al.  A set of 254 Snodgrass-Vanderwart pictures standardized for Spanish: Norms for name agreement, image agreement, familiarity, and visual complexity , 1996 .

[46]  Joseph Douglas Horton,et al.  A Polynomial-Time Algorithm to Find the Shortest Cycle Basis of a Graph , 1987, SIAM J. Comput..

[47]  Marc Alexa,et al.  An evaluation of descriptors for large-scale image retrieval from sketched feature lines , 2010, Comput. Graph..

[48]  Dengsheng Zhang,et al.  An integrated approach to shape based image retrieval , 2002 .

[49]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[50]  Christian Wolf,et al.  3D Object Detection and Viewpoint Selection in Sketch Images Using Local Patch-Based Zernike Moments , 2009, 2009 Seventh International Workshop on Content-Based Multimedia Indexing.

[51]  Ming Ouhyoung,et al.  3D Model Search Engine Based on Lightfield Descriptors , 2003, Eurographics.

[52]  Kun Zhou,et al.  Discriminative Sketch‐based 3D Model Retrieval via Robust Shape Matching , 2011, Comput. Graph. Forum.

[53]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. , 1980, Journal of experimental psychology. Human learning and memory.

[54]  Tobias Schreck,et al.  STELA: sketch-based 3D model retrieval using a structure-based local approach , 2011, ICMR '11.

[55]  Marc Alexa,et al.  Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors , 2011, IEEE Transactions on Visualization and Computer Graphics.

[56]  Max J. Egenhofer,et al.  Reasoning about Gradual Changes of Topological Relationships , 1992, Spatio-Temporal Reasoning.

[57]  Xiaoou Tang,et al.  Example-based 3D object reconstruction from line drawings , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Guojun Lu,et al.  Shape-based image retrieval using generic Fourier descriptor , 2002, Signal Process. Image Commun..

[59]  Adam Finkelstein,et al.  Suggestive contours for conveying shape , 2003, ACM Trans. Graph..

[60]  Joaquim A. Jorge,et al.  CALI: An Online Scribble Recognizer for Calligraphic Interfaces , 2002 .

[61]  Ryutarou Ohbuchi,et al.  Dense sampling and fast encoding for 3D model retrieval using bag-of-visual features , 2009, CIVR '09.

[62]  Bo Li,et al.  Semantic sketch-based 3D model retrieval , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[63]  Thomas Ottmann,et al.  Algorithms for Reporting and Counting Geometric Intersections , 1979, IEEE Transactions on Computers.

[64]  Maciej M. Syslo,et al.  An Efficient Cycle Vector Space Algorithm for Listing all Cycles of a Planar Graph , 1981, SIAM J. Comput..

[65]  Pavel Zezula,et al.  Similarity Search - The Metric Space Approach , 2005, Advances in Database Systems.

[66]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[67]  Hans-Peter Seidel,et al.  Ridge-Valley Lines on Meshes via Implicit Surface Fitting , 2004 .

[68]  Seah Hock Soon,et al.  Efficient and robust 3D line drawings using difference-of-Gaussian , 2012, Graph. Model..

[69]  Hong Qin,et al.  An Effective Illustrative Visualization Framework Based on Photic Extremum Lines (PELs) , 2007, IEEE Transactions on Visualization and Computer Graphics.

[70]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[71]  Joaquim A. Jorge,et al.  Sketch-based Retrieval of Vector Drawings , 2011, Sketch-based Interfaces and Modeling.

[72]  Masaki Aono,et al.  3D shape retrieval from a 2D image as query , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[73]  Xiaoou Tang,et al.  Object cut: Complex 3D object reconstruction through line drawing separation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[74]  S Pompéia,et al.  A set of 400 pictures standardised for Portuguese: norms for name agreement, familiarity and visual complexity for children and adults. , 2001, Arquivos de neuro-psiquiatria.

[75]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[76]  Ilan Shimshoni,et al.  Demarcating curves for shape illustration , 2008, SIGGRAPH 2008.

[77]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[78]  Marc Alexa,et al.  Sketch-based 3D shape retrieval , 2010, SIGGRAPH '10.

[79]  Adam Finkelstein,et al.  Where do people draw lines , 2008, SIGGRAPH 2008.

[80]  Douglas DeCarlo,et al.  Depicting 3D shape using lines , 2012, Electronic Imaging.

[81]  F X Alario,et al.  A set of 400 pictures standardized for French: Norms for name agreement, image agreement, familiarity, visual complexity, image variability, and age of acquisition , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.