Efficient Relevance Feedback for Content-Based Image Retrieval by Mining User Navigation Patterns

Nowadays, content-based image retrieval (CBIR) is the mainstay of image retrieval systems. To be more profitable, relevance feedback techniques were incorporated into CBIR such that more precise results can be obtained by taking user's feedbacks into account. However, existing relevance feedback-based CBIR methods usually request a number of iterative feedbacks to produce refined search results, especially in a large-scale image database. This is impractical and inefficient in real applications. In this paper, we propose a novel method, Navigation-Pattern-based Relevance Feedback (NPRF), to achieve the high efficiency and effectiveness of CBIR in coping with the large-scale image data. In terms of efficiency, the iterations of feedback are reduced substantially by using the navigation patterns discovered from the user query log. In terms of effectiveness, our proposed search algorithm NPRFSearch makes use of the discovered navigation patterns and three kinds of query refinement strategies, Query Point Movement (QPM), Query Reweighting (QR), and Query Expansion (QEX), to converge the search space toward the user's intention effectively. By using NPRF method, high quality of image retrieval on RF can be achieved in a small number of feedbacks. The experimental results reveal that NPRF outperforms other existing methods significantly in terms of precision, coverage, and number of feedbacks.

[1]  Bir Bhanu,et al.  Integrating relevance feedback techniques for image retrieval using reinforcement learning , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Tao Qin,et al.  An active feedback framework for image retrieval , 2008, Pattern Recognit. Lett..

[3]  Christos Faloutsos,et al.  MindReader: Querying Databases Through Multiple Examples , 1998, VLDB.

[4]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[5]  Shih-Fu Chang,et al.  VisualSEEk: a fully automated content-based image query system , 1997, MULTIMEDIA '96.

[6]  Sharad Mehrotra,et al.  Query reformulation for content based multimedia retrieval in MARS , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[7]  Ronald Fagin,et al.  Combining Fuzzy Information from Multiple Systems , 1999, J. Comput. Syst. Sci..

[8]  Vincent S. Tseng,et al.  Web image annotation by fusing visual features and textual information , 2007, SAC '07.

[9]  Gerard Salton,et al.  Improving retrieval performance by relevance feedback , 1997, J. Am. Soc. Inf. Sci..

[10]  Christos Faloutsos,et al.  FALCON: Feedback Adaptive Loop for Content-Based Retrieval , 2000, VLDB.

[11]  Kien A. Hua,et al.  Improving image retrieval effectiveness in query-by-example environment , 2003, SAC '03.

[12]  Ronald Fagin,et al.  Fuzzy queries in multimedia database systems , 1998, PODS '98.

[13]  Zi Huang,et al.  Speed up interactive image retrieval , 2008, The VLDB Journal.

[14]  Thomas S. Huang,et al.  Content-based image retrieval with relevance feedback in MARS , 1997, Proceedings of International Conference on Image Processing.

[15]  Kriengkrai Porkaew,et al.  Query refinement for multimedia similarity retrieval in MARS , 1999, MULTIMEDIA '99.

[16]  Deok-Hwan Kim,et al.  QCluster: relevance feedback using adaptive clustering for content-based image retrieval , 2003, SIGMOD '03.

[17]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[18]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[19]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[20]  James C. French,et al.  Improving Image Retrieval Effectiveness via Multiple Queries , 2003, MMDB '03.

[21]  Donna K. Harman,et al.  Relevance feedback revisited , 1992, SIGIR '92.

[22]  Vincent S. Tseng,et al.  Integrated Mining of Visual Features, Speech Features, and Frequent Patterns for Semantic Video Annotation , 2008, IEEE Transactions on Multimedia.

[23]  Jing Liu,et al.  Human behaviour consistent relevance feedback model for image retrieval , 2007, ACM Multimedia.

[24]  Edward Y. Chang,et al.  Nnew: nearest neighbor expansion by weighting in image database retrieval , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[25]  Alex Pentland,et al.  Photobook: Content-based manipulation of image databases , 1996, International Journal of Computer Vision.

[26]  James C. French,et al.  An Empirical Investigation of the Scalability of a Multiple Viewpoint CBIR System , 2004, CIVR.

[27]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.