Sketch-Based Retrieval Using Content-Aware Hashing

In this paper, we introduce a generic hashing-based approach. It aims to facilitate sketch-based retrieval on large datasets of visual shapes. Unlike previous methods where visual descriptors are extracted from overlapping grids, a content-aware selection scheme is proposed to generate candidate patches instead. Meanwhile, the saliency of each patch is efficiently estimated. Locality-sensitive hashing LSH is employed to integrate and capture both the content and saliency of patches, as well as the spatial information of visual shapes. Furthermore, hash codes are indexed so that a query can be processed in sub-linear time. Experiments on three standard datasets in terms of hand drawn shapes, images and 3D models demonstrate the superiority of our approach.

[1]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark (Figures 1 and 2) , 2004, Shape Modeling International Conference.

[2]  Benjamin Bustos,et al.  Sketch-based image retrieval using keyshapes , 2013, Multimedia Tools and Applications.

[3]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[4]  Changhu Wang,et al.  Indexing billions of images for sketch-based retrieval , 2013, ACM Multimedia.

[5]  Jian Sun,et al.  Saliency Optimization from Robust Background Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Shuang Liang,et al.  Sketch retrieval and relevance feedback with biased SVM classification , 2008, Pattern Recognit. Lett..

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  Bin Li,et al.  Sketch retrieval based on spatial relations , 2005, International Conference on Computer Graphics, Imaging and Visualization (CGIV'05).

[9]  Cordelia Schmid,et al.  Scale & Affine Invariant Interest Point Detectors , 2004, International Journal of Computer Vision.

[10]  David J. Fleet,et al.  Fast search in Hamming space with multi-index hashing , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[12]  Joaquim A. Jorge,et al.  Sketch-based Interfaces and Modeling , 2010 .

[13]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[14]  Thomas A. Funkhouser,et al.  The Princeton Shape Benchmark , 2004, Proceedings Shape Modeling Applications, 2004..

[15]  Moses Charikar,et al.  Similarity estimation techniques from rounding algorithms , 2002, STOC '02.

[16]  Ming-Hsuan Yang,et al.  Sketch Retrieval via Dense Stroke Features , 2013, BMVC.

[17]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Marc Alexa,et al.  Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors , 2011, IEEE Transactions on Visualization and Computer Graphics.

[19]  Ebroul Izquierdo,et al.  Large Scale Sketch Based Image Retrieval Using Patch Hashing , 2012, ISVC.

[20]  Yongdong Zhang,et al.  Binary Code Ranking with Weighted Hamming Distance , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Antti Oulasvirta,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[22]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[23]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Lei Zhang,et al.  MindFinder: A Sketch-based Image Search Engine based on Edgel Index (demo) , 2011 .

[25]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Ryutarou Ohbuchi,et al.  Visual Saliency Weighting and Cross-Domain Manifold Ranking for Sketch-Based Image Retrieval , 2014, MMM.

[27]  Joaquim A. Jorge,et al.  Sketch-based Retrieval of Vector Drawings , 2011, Sketch-based Interfaces and Modeling.

[28]  Chong-Wah Ngo,et al.  Multimedia modeling , 2013, Multimedia Tools and Applications.

[29]  Arjan Kuijper,et al.  Sketch-based 3D model retrieval using diffusion tensor fields of suggestive contours , 2010, ACM Multimedia.