Improving Text-Based Image Search with Textual and Visual Features Combination

With the huge number of available images on the web, an effective image retrieval system has been more and more needed. Improving the performance is one of crucial tasks in modern text-based image retrieval systems such as Google Image Search, Frickr, etc. In this paper, we propose a unified framework to cluster and re-rank returned images with respect to an input query. However, owning to a difference to previous methods of using only either textual or visual features of an image, we combine the textual and visual features to improve search performance. The experimental results show that our proposed model can significantly improve the performance of a text-based image search system (i.e. Flickr). Moreover, the performance of the system with the combination of textual and visual features outperforms the performance of both the textual-based system and the visual-based system.

[1]  Hideyuki Tamura,et al.  Image database systems: A survey , 1984, Pattern Recognit..

[2]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[3]  Nick Craswell,et al.  Visual clustering of image search results , 2001, IS&T/SPIE Electronic Imaging.

[4]  Thomas Deselaers,et al.  Clustering visually similar images to improve image search engines , 2003 .

[5]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[6]  Amarnath Gupta,et al.  Visual information retrieval , 1997, CACM.

[7]  N. Ikizler,et al.  Re-ranking of image search results using a graph algorithm , 2008, 2008 IEEE 16th Signal Processing, Communication and Applications Conference.

[8]  Xiaogang Wang,et al.  World Wide Web Based Image Search Engine Using Text and Image Content Features , 2003, IS&T/SPIE Electronic Imaging.

[9]  Jan-Ming Ho,et al.  A novel content based image retrieval system using K-means with feature extraction , 2012, 2012 International Conference on Systems and Informatics (ICSAI2012).

[10]  Byungwook Lee,et al.  A color-based clustering approach for web image search results , 2009, ICHIT '09.

[11]  Hideyuki Tamura,et al.  Textural Features Corresponding to Visual Perception , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[12]  Hermann Ney,et al.  Features for image retrieval: an experimental comparison , 2008, Information Retrieval.

[13]  Jiebo Luo,et al.  RankCompete: simultaneous ranking and clustering of web photos , 2010, WWW '10.

[14]  Sougata Mukherjea,et al.  Using clustering and visualization for refining the results of a WWW image search engine , 1998, NPIV '98.

[15]  Wei-Ying Ma,et al.  Grouping web image search result , 2004, MULTIMEDIA '04.

[16]  Jiebo Luo,et al.  Data Mining. Multimedia, Soft Computing, and Bioinformatics , 2005, IEEE Transactions on Neural Networks.

[17]  Jing Liu,et al.  Hierarchical clustering-based navigation of image search results , 2008, ACM Multimedia.

[18]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[19]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Shi-Kuo Chang,et al.  Image Information Systems: Where Do We Go From Here? , 1992, IEEE Trans. Knowl. Data Eng..

[21]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.