Relevance Feedback for Keyword and Visual Feature-Based Image Retrieval

In this paper, a relevance feedback scheme for both keyword and visual feature-based image retrieval is proposed. For each keyword, a statistical model is trained offline based on visual features of a small set of manually labeled images and used to propagate the keyword to other unlabeled ones. Besides the offline model, another model is constructed online using the user provided positive and negative images as training set. Support vector machines (SVMs) in the binary setting are adopted as both offline and online models. To effectively combine the two models, a multi-model query refinement algorithm is introduced. Furthermore, an entropy-based active learning strategy is proposed to improve the efficiency of relevance feedback process. Experimental results on a database of 10,000 general-purpose images demonstrate the effectiveness of the proposed relevance feedback scheme.

[1]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[2]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[3]  Tsuhan Chen,et al.  Indexing and retrieval of 3D models aided by active learning , 2001, MULTIMEDIA '01.

[4]  Hideyuki Tamura,et al.  Image database systems: A survey , 1984, Pattern Recognit..

[5]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[6]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[7]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[8]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[9]  Gerald Salton,et al.  Automatic text processing , 1988 .

[10]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[11]  Gunnar Rätsch,et al.  Active Learning in the Drug Discovery Process , 2001, NIPS.

[12]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[13]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[14]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Vapnik,et al.  SVMs for Histogram Based Image Classification , 1999 .

[17]  Beng Chin Ooi,et al.  Giving meanings to WWW images , 2000, ACM Multimedia.

[18]  H. J. Zhang,et al.  IMPROVING CBIR BY SEMANTIC PROPAGATION AND CROSS MODALITY QUERY EXPANSION , 2001 .