Combining conceptual query expansion and visual search results exploration for web image retrieval

Most approaches to image retrieval on the Web have their basis in document search techniques. Images are indexed based on the text that is related to the images. Queries are matched to this text to produce a set of search results, which are organized in paged grids that are reminiscent of lists of documents. Due to ambiguity both with the user-supplied query and with the text used to describe the images within the search index, most image searches contain many irrelevant images distributed throughout the search results. In this paper we present a method for addressing this problem.We perform conceptual query expansion using Wikipedia in order to generate a diverse range of images for each query, and then use a multi-resolution self organizing map to group visually similar images. The resulting interface acts as an intelligent search assistant, automatically diversifying the search results and then allowing the searcher to interactively highlight and filter images based on the concepts, and zoom into an area within the image space to show additional images that are visually similar.

[1]  Jens Lehmann,et al.  DBpedia - A crystallization point for the Web of Data , 2009, J. Web Semant..

[2]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  James Ze Wang,et al.  PARAgrab: a comprehensive architecture for web image management and multimodal querying , 2006, VLDB.

[4]  Wei-Ying Ma,et al.  Image and Video Retrieval , 2003, Lecture Notes in Computer Science.

[5]  Djemel Ziou,et al.  Image Retrieval from the World Wide Web: Issues, Techniques, and Systems , 2004, CSUR.

[6]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[7]  Daniel Heesch,et al.  A survey of browsing models for content based image retrieval , 2008, Multimedia Tools and Applications.

[8]  Adrian Popescu,et al.  Multimodal Image Retrieval over a Large Database , 2009, CLEF.

[9]  Ian H. Witten,et al.  Learning to link with wikipedia , 2008, CIKM '08.

[10]  Minglun Gong,et al.  Visual Image Browsing and Exploration (Vibe): User Evaluations of Image Search Tasks , 2010, AMT.

[11]  Ian H. Witten,et al.  An effective, low-cost measure of semantic relatedness obtained from Wikipedia links , 2008 .

[12]  Desney S. Tan,et al.  Designing Novel Image Search Interfaces by Understanding Unique Characteristics and Usage , 2009, INTERACT.

[13]  Peter G. B. Enser,et al.  Towards a Comprehensive Survey of the Semantic Gap in Visual Image Retrieval , 2003, CIVR.

[14]  Minglun Gong,et al.  Browsing a Large Collection of Community Photos Based on Similarity on GPU , 2008, ISVC.

[15]  Berthier A. Ribeiro-Neto,et al.  Concept-based interactive query expansion , 2005, CIKM '05.

[16]  Minglun Gong,et al.  Organizing and browsing photos using different feature vectors and their evaluations , 2009, CIVR '09.

[17]  Simone Paolo Ponzetto,et al.  WikiRelate! Computing Semantic Relatedness Using Wikipedia , 2006, AAAI.

[18]  Evgeniy Gabrilovich,et al.  Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis , 2007, IJCAI.

[19]  Ian H. Witten,et al.  A knowledge-based search engine powered by wikipedia , 2007, CIKM '07.

[20]  Amanda Spink,et al.  An analysis of multimedia searching on AltaVista , 2003, MIR '03.

[21]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.