Multi-model similarity propagation and its application for web image retrieval

In this paper, we propose an iterative similarity propagation approach to explore the inter-relationships between Web images and their textual annotations for image retrieval. By considering Web images as one type of objects, their surrounding texts as another type, and constructing the links structure between them via webpage analysis, we can iteratively reinforce the similarities between images. The basic idea is that if two objects of the same type are both related to one object of another type, these two objects are similar; likewise, if two objects of the same type are related to two different, but similar objects of another type, then to some extent, these two objects are also similar. The goal of our method is to fully exploit the mutual reinforcement between images and their textual annotations. Our experiments based on 10,628 images crawled from the Web show that our proposed approach can significantly improve Web image retrieval performance.

[1]  Thomas S. Huang,et al.  Image processing , 1971 .

[2]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[3]  Michel Barlaud,et al.  Image coding using wavelet transform , 1992, IEEE Trans. Image Process..

[4]  John P. Oakley,et al.  Storage and Retrieval for Image and Video Databases , 1993 .

[5]  Ramesh Jain,et al.  Storage and Retrieval for Image and Video Databases III , 1995 .

[6]  Markus A. Stricker,et al.  Similarity of color images , 1995, Electronic Imaging.

[7]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  S. Sclaroff,et al.  Combining textual and visual cues for content-based image retrieval on the World Wide Web , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[9]  Mark Craven,et al.  Combining Statistical and Relational Methods for Learning in Hypertext Domains , 1998, ILP.

[10]  G. Erald Duung.  Text-Image Interaction for Image Retrieval and Semi-Automatic Indexing , 1998 .

[11]  Jon M. Kleinberg,et al.  Inferring Web communities from link topology , 1998, HYPERTEXT '98.

[12]  M. KleinbergJon.  Authoritative sources in a hyperlinked environment , 1999 .

[13]  Rohini K. Srihari,et al.  A model for multimodal information retrieval , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[14]  Jennifer Neville,et al.  Iterative Classification in Relational Data , 2000 .

[15]  Mingjing Li,et al.  Web mining for Web image retrieval , 2001, J. Assoc. Inf. Sci. Technol..

[16]  David A. Forsyth,et al.  Clustering art , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Nello Cristianini,et al.  Learning Semantic Similarity , 2002, NIPS.

[18]  Thijs Westerveld.  Probabilistic multimedia retrieval , 2002, SIGIR '02.

[19]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[20]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[21]  Hongjun Lu,et al.  ReCoM: reinforcement clustering of multi-type interrelated data objects , 2003, SIGIR.

[22]  Wei-Ying Ma,et al.  Extracting Content Structure for Web Pages Based on Visual Representation , 2003, APWeb.

[23]  Wei-Ying Ma,et al.  VIPS: a Vision-based Page Segmentation Algorithm , 2003 .

[24]  Wei-Ying Ma,et al.  Similarity spreading: a unified framework for similarity calculation of interrelated objects , 2004, WWW Alt. '04.