Combining positive and negative examples in relevance feedback for content-based image retrieval

Abstract In this paper, we address some issues related to the combination of positive and negative examples to improve the efficiency of image retrieval. We start by analyzing the relevance of the negative example and how it can be interpreted and utilized to mitigate certain problems in image retrieval, such as noise, miss, the page zero problem and feature selection. Then we propose a new relevance feedback approach that uses the positive example (PE) to perform generalization and the negative example (NE) to perform specialization. In this approach, a query containing both PE and NE is processed in two steps. The first step considers the PE alone, in order to reduce the set of images participating in retrieval to a more homogeneous subset. Then, the second step considers both PE and NE and acts on the images retained in the first step. Mathematically, relevance feedback is formulated as an optimization of the intra and inter variances of the PE and NE. The proposed relevance feedback algorithm was implemented in our image retrieval system, which we tested on a collection of more than 10,000 images. The experimental results show how the NE as considered in our model can contribute in improving the relevance of the images retrieved.

[1]  Christos Faloutsos,et al.  MindReader: Querying Databases Through Multiple Examples , 1998, VLDB.

[2]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Thomas S. Huang,et al.  Content-based image retrieval with relevance feedback in MARS , 1997, Proceedings of International Conference on Image Processing.

[4]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[5]  J. J. Rocchio,et al.  Relevance feedback in information retrieval , 1971 .

[6]  Bo Zhang,et al.  Region-based relevance feedback in image retrieval , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[7]  Jitendra Malik,et al.  Blobworld: A System for Region-Based Image Indexing and Retrieval , 1999, VISUAL.

[8]  Tom Minka,et al.  Interactive learning with a "Society of Models" , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Dragutin Petkovic,et al.  Query by Image and Video Content: The QBIC System , 1995, Computer.

[10]  Joshua R. Smith,et al.  Image retrieval evaluation , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[11]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Ryszard S. Michalski,et al.  A Theory and Methodology of Inductive Learning , 1983, Artificial Intelligence.

[13]  Thomas S. Huang,et al.  Optimizing learning in image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Shih-Fu Chang,et al.  Tools and techniques for color image retrieval , 1996, Electronic Imaging.

[15]  James Ze Wang,et al.  IRM: integrated region matching for image retrieval , 2000, ACM Multimedia.

[16]  Bir Bhanu,et al.  Learning feature relevance and similarity metrics in image databases , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[17]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[18]  Chahab Nastar,et al.  Efficient query refinement for image retrieval , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[19]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[20]  Yong Rui,et al.  Efficient Indexing, Browsing and Retrieval of Image/Video Content , 1999 .

[21]  Thomas S. Huang,et al.  Comparing discriminating transformations and SVM for learning during multimedia retrieval , 2001, MULTIMEDIA '01.

[22]  Nicholas J. Belkin,et al.  Rutgers' TREC 2001 Interactive Track Experience , 2001, TREC.

[23]  Rosalind W. Picard,et al.  Interactive Learning Using a "Society of Models" , 2017, CVPR 1996.

[24]  Thomas S. Huang,et al.  Evaluating group-based relevance feedback for content-based image retrieval , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[25]  Ingemar J. Cox,et al.  PicHunter: Bayesian relevance feedback for image retrieval , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[26]  Filippo Menczer,et al.  Feature selection in unsupervised learning via evolutionary search , 2000, KDD '00.

[27]  Chahab Nastar,et al.  Relevance feedback and category search in image databases , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[28]  Thierry Pun,et al.  Strategies for positive and negative relevance feedback in image retrieval , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[29]  HongJiang Zhang.  Relevance Feedback in Content-based Image Search , 2001, PRIS.

[30]  Nuno Vasconcelos,et al.  Learning from User Feedback in Image Retrieval Systems , 1999, NIPS.

[31]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Alain Trouvé,et al.  Metric Similarities Learning through Examples: An Application to Shape Retrieval , 2001, EMMCVPR.

[33]  Tom Minka,et al.  Modeling user subjectivity in image libraries , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[34]  Jiawei Han,et al.  Knowledge Discovery in Databases: An Attribute-Oriented Approach , 1992, VLDB.

[35]  Marco La Cascia,et al.  Image Digestion and Relevance Feedback in the ImageRover WWW Search Engine , 1997 .