Relevance feedback in image retrieval: A comprehensive review

Abstract. We analyze the nature of the relevance feedback problem in a continuous representation space in the context of content-based image retrieval. Emphasis is put on exploring the uniqueness of the problem and comparing the assumptions, implementations, and merits of various solutions in the literature. An attempt is made to compile a list of critical issues to consider when designing a relevance feedback algorithm. With a comprehensive review as the main portion, this paper also offers some novel solutions and perspectives throughout the discussion.

[1]  Toshikazu Kato,et al.  Learning of personal visual impression for image database systems , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[2]  Paul A. Viola,et al.  Boosting Image Retrieval , 2004, International Journal of Computer Vision.

[3]  Rosalind W. Picard,et al.  Interactive Learning Using a "Society of Models" , 2017, CVPR 1996.

[4]  Yaowu Xu,et al.  Hierarchical content description and object formation by learning , 1999, Proceedings IEEE Workshop on Content-Based Access of Image and Video Libraries (CBAIVL'99).

[5]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[6]  David A. Forsyth,et al.  Finding people and animals by guided assembly , 1997, Proceedings of International Conference on Image Processing.

[7]  Daphne Koller,et al.  Support Vector Machine Active Learning with Application sto Text Classification , 2000, ICML.

[8]  Qi Tian,et al.  Incorporate support vector machines to content-based image retrieval with relevance feedback , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[9]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[10]  Ingemar J. Cox,et al.  An optimized interaction strategy for Bayesian relevance feedback , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[11]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Thomas S. Huang,et al.  Small sample learning during multimedia retrieval using BiasMap , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Christos Faloutsos,et al.  FALCON: Feedback Adaptive Loop for Content-Based Retrieval , 2000, VLDB.

[14]  Sharad Mehrotra,et al.  Query reformulation for content based multimedia retrieval in MARS , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[15]  Neill W. Campbell,et al.  Iterative refinement by relevance feedback in content-based digital image retrieval , 1998, MULTIMEDIA '98.

[16]  Thomas S. Huang,et al.  Generalized relevance feedback scheme for image retrieval , 2000, SPIE Optics East.

[17]  Tom Minka,et al.  Interactive learning with a "Society of Models" , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Tom Minka,et al.  Modeling user subjectivity in image libraries , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[19]  Daphne Koller,et al.  Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..

[20]  Christos Faloutsos,et al.  MindReader: Querying Databases Through Multiple Examples , 1998, VLDB.

[21]  S. Sclaroff,et al.  Combining textual and visual cues for content-based image retrieval on the World Wide Web , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[22]  Thomas S. Huang,et al.  ICA-based probabilistic local appearance models , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[23]  Simone Santini,et al.  Integrated browsing and querying for image databases , 2000, IEEE MultiMedia.

[24]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[25]  Thomas S. Huang,et al.  Optimizing learning in image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[26]  B. S. Manjunath,et al.  Adaptive nearest neighbor search for relevance feedback in large image databases , 2001, MULTIMEDIA '01.

[27]  Qi Tian,et al.  Discriminant-EM algorithm with application to image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Erkki Oja,et al.  PicSOM: self-organizing maps for content-based image retrieval , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[29]  Stan Z. Li,et al.  Extraction of feature subspaces for content-based retrieval using relevance feedback , 2001, MULTIMEDIA '01.

[30]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[31]  King-Sun Fu,et al.  Error-Correcting Isomorphisms of Attributed Relational Graphs for Pattern Analysis , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[32]  Marco La Cascia,et al.  Unifying Textual and Visual Cues for Content-Based Image Retrieval on the World Wide Web , 1999, Comput. Vis. Image Underst..

[33]  Chahab Nastar,et al.  Efficient query refinement for image retrieval , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[34]  William A. Gale,et al.  A sequential algorithm for training text classifiers , 1994, SIGIR '94.

[35]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[36]  L. R. Haff.  Empirical Bayes Estimation of the Multivariate Normal Covariance Matrix , 1980 .

[37]  Thomas S. Huang,et al.  Spatial pattern discovering by learning the isomorphic subgraph from multiple attributed relation graphs , 2001, Electron. Notes Theor. Comput. Sci..

[38]  Nuno Vasconcelos,et al.  Learning from User Feedback in Image Retrieval Systems , 1999, NIPS.

[39]  Fabio Roli,et al.  Bayesian relevance feedback for content-based image retrieval , 2004, Pattern Recognit..

[40]  Hans-Jörg Schek,et al.  Interactive-Time Similarity Search for Large Image Collections Using Parallel VA-Files , 2000, ECDL.

[41]  W. Eric L. Grimson,et al.  A framework for learning query concepts in image classification , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[42]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[43]  Marcel Worring,et al.  Interaction in Content-Based Image Retrieval: The Evaluation of the State-of-the-Art Review , 2000, VISUAL.

[44]  Robert M. Haralick,et al.  Probabilistic vs. geometric similarity measures for image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[45]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[46]  Thomas S. Huang,et al.  Visualization and Layout for Personal Photo Libraries , 2001 .

[47]  Xiang Zhou,et al.  Factorized local appearance models , 2002, Object recognition supported by user interaction for service robots.

[48]  David G. Lowe,et al.  Similarity Metric Learning for a Variable-Kernel Classifier , 1995, Neural Computation.

[49]  Raimondo Schettini,et al.  Content-based color image retrieval with relevance feedback , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[50]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[51]  Xiang Sean Zhou,et al.  Image retrieval: feature primitives, feature representation, and relevance feedback , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[52]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[53]  Bir Bhanu,et al.  Probabilistic Feature Relevance Learning for Content-Based Image Retrieval , 1999, Comput. Vis. Image Underst..

[54]  Roberto Brunelli,et al.  Image Retrieval by Examples , 2000, IEEE Trans. Multim..

[55]  John C. Dalton,et al.  Hierarchical browsing and search of large image databases , 2000, IEEE Trans. Image Process..

[56]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[57]  Chahab Nastar,et al.  Relevance feedback and category search in image databases , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[58]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[59]  Alberto Del Bimbo.  Guest Editor's Introduction: Multimedia Computing and Systems , 2000, IEEE Multim..

[60]  Guodong Guo,et al.  Boosting for content-based audio classification and retrieval: an evaluation , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[61]  Edward Y. Chang,et al.  Learning image query concepts via intelligent sampling , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[62]  Ilaria Bartolini,et al.  FeedbackBypass: A New Approach to Interactive Similarity Query Processing , 2001, VLDB.

[63]  Chi-Ren Shyu,et al.  Relevance feedback decision trees in content-based image retrieval , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[64]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.

[65]  Gerald Salton,et al.  Automatic text processing , 1988 .

[66]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[67]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[68]  Wei-Ying Ma,et al.  Information embedding based on user's relevance feedback for image retrieval , 1999, Optics East.

[69]  Jing Xin,et al.  Learning from user feedback for image retrieval , 2003, Fourth International Conference on Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint.