Graph-Cut Transducers for Relevance Feedback in Content Based Image Retrieval

Closing the semantic gap in content based image retrieval (CBIR) basically requires the knowledge of the user's intention which is usually translated into a sequence of questions and answers (Q&A). The user's feedback to these questions provides a CBIR system with a partial labeling of the data and makes it possible to iteratively refine a decision rule on the unlabeled data. Training of this decision rule is referred to as transductive learning. This work is an original approach to relevance feedback (RF) based on graph-cuts. Training consists in implicitly modeling the manifold enclosing both the labeled and unlabeled dataset and finding a partition of this manifold using a min-cut. The contribution of this work is two-fold (i) this is the first comprehensive study of relevance feedback using graph cuts and (ii) our RF model exploits the structure of the data manifold by considering also the structure of the unlabeled data. Experiments conducted on generic as well as specific databases show that our graph-cut based approach is very effective, outperforms other existing methods and makes it possible to converge to almost all the images of the user's "class of interest" with a very small labeling effort. A demo is available through our image retrieval tool kit (IRTK).

[1]  Thomas Hofmann,et al.  Active learning for misspecified generalized linear models , 2007 .

[2]  Toshikazu Kato,et al.  Learning of personal visual impression for image database systems , 1993, Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR '93).

[3]  Paul A. Viola,et al.  Boosting Image Retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[5]  Donald Geman,et al.  A Stochastic Feedback Model for Image Retrieval , 1999 .

[6]  Olivier Juan,et al.  Active Graph Cuts , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Christos Faloutsos,et al.  MindReader: Querying Databases Through Multiple Examples , 1998, VLDB.

[8]  Hichem Sahbi,et al.  Kernel PCA for similarity invariant shape recognition , 2007, Neurocomputing.

[9]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[10]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[12]  Ingemar J. Cox,et al.  An optimized interaction strategy for Bayesian relevance feedback , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[13]  Yao Zhao,et al.  Relevance Feedback based on Query Refining and Feature Database Updating in CBIR System , 2006, SPPRA.

[14]  Thomas S. Huang,et al.  Relevance feedback techniques in interactive content-based image retrieval , 1997, Electronic Imaging.

[15]  Chi-Ren Shyu,et al.  Relevance feedback decision trees in content-based image retrieval , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[16]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[17]  Michael Ian Shamos,et al.  Computational geometry: an introduction , 1985 .

[18]  F. Fleuret,et al.  Scale-Invariance of Support Vector Machines based on the Triangular Kernel , 2001 .

[19]  Tom Minka,et al.  Modeling user subjectivity in image libraries , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[20]  Pushmeet Kohli,et al.  Measuring Uncertainty in Graph Cut Solutions - Efficiently Computing Min-marginal Energies Using Dynamic Graph Cuts , 2006, ECCV.

[21]  Yuchun Fang,et al.  Experiments in Mental Face Retrieval , 2005, AVBPA.

[22]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[23]  Greg Schohn,et al.  Less is More: Active Learning with Support Vector Machines , 2000, ICML.

[24]  Nuno Vasconcelos,et al.  Learning from User Feedback in Image Retrieval Systems , 1999, NIPS.

[25]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[26]  D. R. Fulkerson,et al.  Flows in Networks. , 1964 .

[27]  Chahab Nastar,et al.  Relevance feedback in Surfimage , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[28]  Bernhard Schölkopf,et al.  On the Relation Between Low Density Separation, Spectral Clustering and Graph Cuts , 2007 .

[29]  Marin Ferecatu,et al.  Retrieval of difficult image classes using svd-based relevance feedback , 2004, MIR '04.

[30]  Avrim Blum,et al.  Learning from Labeled and Unlabeled Data using Graph Mincuts , 2001, ICML.

[31]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[33]  Mikhail Belkin,et al.  On the Relation Between Low Density Separation, Spectral Clustering and Graph Cuts , 2006, NIPS.

[34]  Eric J. Pauwels,et al.  Logistic regression model for relevance feedback in content-based image retrieval , 2001, IS&T/SPIE Electronic Imaging.

[35]  Kwang In Kim,et al.  Face recognition using kernel principal component analysis , 2002, IEEE Signal Processing Letters.