When Amazon Meets Google: Product Visualization by Exploring Multiple Web Sources

Product visualization is able to help users easily get knowledge about the visual appearance of a product. It is useful in many application and commercialization scenarios. However, the existing product image search on e-commerce Web sites or general search engines usually get insufficient search results or return images that are redundant and not relevant enough. In this article, we present a novel product visualization approach that automatically collects a set of diverse and relevant product images by exploring multiple Web sources. Our approach simultaneously leverages Amazon and Google image search engines, which represent domain-specific knowledge resource and general Web information collection, respectively. We propose a conditional clustering approach that is formulated as an affinity propagation problem regarding the Amazon examples as information prior. The ranking information of Google image search results is also explored. In this way, a set of exemplars can be found from the Google search results and they are provided together with the Amazon example images for product visualization. Experiments demonstrate the feasibility and effectiveness of our approach.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Yue Gao,et al.  View-Based Discriminative Probabilistic Modeling for 3D Object Retrieval and Recognition , 2013, IEEE Transactions on Image Processing.

[3]  Ximena Olivares,et al.  Visual diversification of image search results , 2009, WWW '09.

[4]  Hua Li,et al.  Mobile Search With Multimodal Queries , 2008, Proceedings of the IEEE.

[5]  Hermann Ney,et al.  Jointly optimising relevance and diversity in image retrieval , 2009, CIVR '09.

[6]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[7]  Bingbing Ni,et al.  Assistive tagging: A survey of multimedia tagging with human-computer joint exploration , 2012, CSUR.

[8]  Neil J. Hurley,et al.  Novelty and Diversity in Top-N Recommendation -- Analysis and Evaluation , 2011, TOIT.

[9]  Hao Xu,et al.  Hybrid image summarization , 2011, ACM Multimedia.

[10]  Rui Liu,et al.  Image Search Result Summarization with Informative Priors , 2009, ACCV.

[11]  Xian-Sheng Hua,et al.  Towards a Relevant and Diverse Search of Social Images , 2010, IEEE Transactions on Multimedia.

[12]  E. Minium.  Statistical reasoning in psychology and education , 1970 .

[13]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[14]  Hervé Jégou,et al.  A Group Testing Framework for Similarity Search in High-dimensional Spaces , 2014, ACM Multimedia.

[15]  Shih-Fu Chang,et al.  A Content-Based Image Meta-Search Engine using Relevance Feedback , 1998 .

[16]  Shumeet Baluja,et al.  Pagerank for product image search , 2008, WWW.

[17]  Wei-Ying Ma,et al.  IGroup: web image search results clustering , 2006, MM '06.

[18]  Brendan J. Frey,et al.  Non-metric affinity propagation for unsupervised image categorization , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Bernd Girod,et al.  Mobile product recognition , 2010, ACM Multimedia.

[21]  Shih-Fu Chang,et al.  Mobile product search with Bag of Hash Bits and boundary reranking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Ramayya Krishnan,et al.  Privacy-preserving similarity-based text retrieval , 2010, TOIT.

[23]  Brendan J. Frey,et al.  A Binary Variable Model for Affinity Propagation , 2009, Neural Computation.

[24]  Xuelong Li,et al.  Multitraining Support Vector Machine for Image Retrieval , 2006, IEEE Transactions on Image Processing.

[25]  Meng Wang,et al.  Visual query suggestion , 2009, ACM Multimedia.

[26]  Xian-Sheng Hua,et al.  Finding image exemplars using fast sparse affinity propagation , 2008, ACM Multimedia.

[27]  Dong Liu,et al.  Semi-Automatic Tagging of Photo Albums via Exemplar Selection and Tag Inference , 2011, IEEE Transactions on Multimedia.

[28]  Nenghai Yu,et al.  iLike: integrating visual and textual features for vertical search , 2010, ACM Multimedia.

[29]  Harry Shum,et al.  Picture Collage , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[30]  Shih-Fu Chang,et al.  Using Relevance Feedback in Content-Based Image Metasearch , 1998, IEEE Internet Comput..

[31]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[32]  Andrew Blake,et al.  Digital tapestry [automatic image synthesis] , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Shih-Fu Chang,et al.  Query-Adaptive Fusion for Multimodal Search , 2008, Proceedings of the IEEE.

[34]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.