A new algorithm for product image search based on salient edge characterization

Visually assisted product image search has gained increasing popularity because of its capability to greatly improve end users' e‐commerce shopping experiences. Different from general‐purpose content‐based image retrieval (CBIR) applications, the specific goal of product image search is to retrieve and rank relevant products from a large‐scale product database to visually assist a user's online shopping experience. In this paper, we explore the problem of product image search through salient edge characterization and analysis, for which we propose a novel image search method coupled with an interactive user region‐of‐interest indication function. Given a product image, the proposed approach first extracts an edge map, based on which contour curves are further extracted. We then segment the extracted contours into fragments according to the detected contour corners. After that, a set of salient edge elements is extracted from each product image. Based on salient edge elements matching and similarity evaluation, the method derives a new pairwise image similarity estimate. Using the new image similarity, we can then retrieve product images. To evaluate the performance of our algorithm, we conducted 120 sessions of querying experiments on a data set comprised of around 13k product images collected from multiple, real‐world e‐commerce websites. We compared the performance of the proposed method with that of a bag‐of‐words method (Philbin, Chum, Isard, Sivic, & Zisserman, 2008) and a Pyramid Histogram of Orientated Gradients (PHOG) method (Bosch, Zisserman, & Munoz, 2007). Experimental results demonstrate that the proposed method improves the performance of example‐based product image retrieval.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  K. Mullen,et al.  How long range is contour integration in human color vision? , 2003, Visual Neuroscience.

[3]  Pietro Perona,et al.  Continuous dynamic time warping for translation-invariant curve alignment with applications to signature verification , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Sang Uk Lee,et al.  Color-Based Image Retrieval Using Perceptually Modified Hausdorff Distance , 2008, EURASIP J. Image Video Process..

[5]  Hao Jiang,et al.  Retrieving and ranking unannotated images through collaboratively mining online search results , 2011, CIKM '11.

[6]  Rajiv Mehrotra,et al.  Similar-Shape Retrieval in Shape Data Management , 1995, Computer.

[7]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[8]  Shumeet Baluja,et al.  Pagerank for product image search , 2008, WWW.

[9]  Rudra Prakash Maheshwari,et al.  HOG feature and vocabulary tree for Content-based Image Retrieval , 2010 .

[10]  Bernard J. Jansen,et al.  The seventeen theoretical constructs of information searching and information retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[11]  James H. Elder,et al.  Image Editing in the Contour Domain , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Jiri Matas,et al.  Learning a Fine Vocabulary , 2010, ECCV.

[13]  Michael Isard,et al.  Bundling features for large scale partial-duplicate web image search , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Fionn Murtagh,et al.  A Survey of Recent Advances in Hierarchical Clustering Algorithms , 1983, Comput. J..

[15]  Thomas S. Huang,et al.  Edge-based structural features for content-based image retrieval , 2001, Pattern Recognit. Lett..

[16]  Farzin Mokhtarian,et al.  Robust Image Corner Detection Through Curvature Scale Space , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Gloria Bordogna,et al.  A flexible content‐based image retrieval model and a customizable system for the retrieval of shapes , 2010, J. Assoc. Inf. Sci. Technol..

[18]  Jiri Matas,et al.  Total recall II: Query expansion revisited , 2011, CVPR 2011.

[19]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[20]  Nelson H. C. Yung,et al.  Corner detector based on global and local curvature properties , 2008 .

[21]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Thomas Deselaers,et al.  Exploring The Relationship Between Feature and Perceptual Visual Spaces , 2022 .

[23]  Michael Isard,et al.  Descriptor Learning for Efficient Retrieval , 2010, ECCV.

[24]  Dusan Cakmakov,et al.  ESTIMATION OF CURVE SIMILARITY USING TURNING FUNCTIONS , 2004 .

[25]  Daniel P. Lopresti,et al.  Feature-based approach for image retrieval by sketch , 1997, Other Conferences.

[26]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Xiaoou Tang,et al.  2D Shape Matching by Contour Flexibility , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Kien A. Hua,et al.  Improving image retrieval effectiveness in query-by-example environment , 2003, SAC '03.

[30]  Donald A. Adjeroh,et al.  Effective invariant features for shape-based image retrieval , 2005, J. Assoc. Inf. Sci. Technol..

[31]  Sharath Pankanti,et al.  The relation between the ROC curve and the CMC , 2005, Fourth IEEE Workshop on Automatic Identification Advanced Technologies (AutoID'05).

[32]  Xiaoou Tang,et al.  IntentSearch: interactive on-line image search re-ranking , 2008, ACM Multimedia.

[33]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[34]  Ian H. Witten,et al.  Managing Gigabytes: Compressing and Indexing Documents and Images , 1999 .

[35]  Xian-Sheng Hua,et al.  Color-Structured Image Search , 2009 .

[36]  Sourav S. Bhowmick,et al.  Tag-based social image retrieval: An empirical evaluation , 2011, J. Assoc. Inf. Sci. Technol..

[37]  Xiaofan Lin,et al.  Visual search engine for product images , 2008, Electronic Imaging.

[38]  Vijay V. Raghavan,et al.  A cluster-based approach for efficient content-based image retrieval using a similarity-preserving space transformation method , 2006 .

[39]  Michael Isard,et al.  Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[40]  Djemel Ziou,et al.  Learning from negative example in relevance feedback for content-based image retrieval , 2002, Object recognition supported by user interaction for service robots.

[41]  Hong-Jiang Zhang,et al.  An efficient and effective region-based image retrieval framework , 2004, IEEE Transactions on Image Processing.

[42]  Changhu Wang,et al.  MindFinder: image search by interactive sketching and tagging , 2010, WWW '10.

[43]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  Kai-Kuang Ma,et al.  Rotation-invariant and scale-invariant Gabor features for texture image retrieval , 2007, Image Vis. Comput..

[45]  Xiaoou Tang,et al.  User intention modeling for interactive image retrieval , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[46]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.