Interactive pattern analysis for relevance feedback in multimedia information retrieval

Abstract.Relevance feedback is a mechanism to interactively learn a user’s query concept online. It has been extensively used to improve the performance of multimedia information retrieval. In this paper, we present a novel interactive pattern analysis method that reduces relevance feedback to a two-class classification problem and classifies multimedia objects as relevant or irrelevant. To perform interactive pattern analysis, we propose two online pattern classification methods, called interactive random forests (IRF) and adaptive random forests (ARF), that adapt a composite classifier known as random forests for relevance feedback. IRF improves the efficiency of regular random forests (RRF) with a novel two-level resampling technique called biased random sample reduction, while ARF boosts the performance of RRF with two adaptive learning techniques called dynamic feature extraction and adaptive sample selection. During interactive multimedia retrieval, both ARF and IRF run two to three times faster than RRF while achieving comparable precision and recall against the latter. Extensive experiments on a COREL image set (with 31,438 images) demonstrate that our methods (i.e., IRF and RRF) achieve at least a $20\%$ improvement on average precision and recall over the state-of-the-art approaches.

[1]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[2]  Wei-Ying Ma,et al.  Learning similarity measure for natural image retrieval with relevance feedback , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Rajeev Motwani,et al.  Incremental Clustering and Dynamic Information Retrieval , 2004, SIAM J. Comput..

[4]  Yimin Wu,et al.  Interactive Patterns Analysis for Searching Multimedia Databases , 2002, Multimedia Information Systems.

[5]  Yimin Wu,et al.  Category-based search using metadatabase in image retrieval , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[6]  Paul A. Viola,et al.  Boosting Image Retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  Thomas S. Huang,et al.  Optimizing learning in image retrieval , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  Thomas S. Huang,et al.  Relevance feedback in image retrieval: A comprehensive review , 2003, Multimedia Systems.

[9]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[10]  Ingemar J. Cox,et al.  The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments , 2000, IEEE Trans. Image Process..

[11]  Christos Faloutsos,et al.  MindReader: Querying Databases Through Multiple Examples , 1998, VLDB.

[12]  Yiming Yang,et al.  A Comparative Study on Feature Selection in Text Categorization , 1997, ICML.

[13]  Kriengkrai Porkaew,et al.  Query refinement for multimedia similarity retrieval in MARS , 1999, MULTIMEDIA '99.

[14]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[15]  Borivoje Furht,et al.  Handbook on Multimedia Computing , 1998 .

[16]  Ilaria Bartolini,et al.  FeedbackBypass: A New Approach to Interactive Similarity Query Processing , 2001, VLDB.

[17]  Chi-Ren Shyu,et al.  Relevance feedback decision trees in content-based image retrieval , 2000, 2000 Proceedings Workshop on Content-based Access of Image and Video Libraries.

[18]  L. Breiman.  Random Forests--random Features , 1999 .

[19]  Shih-Fu Chang,et al.  Transform features for texture classification and discrimination in large image databases , 1994, Proceedings of 1st International Conference on Image Processing.

[20]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[21]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[22]  Tapio Elomaa,et al.  General and Efficient Multisplitting of Numerical Attributes , 1999, Machine Learning.

[23]  Christos Faloutsos,et al.  QBIC project: querying images by content, using color, texture, and shape , 1993, Electronic Imaging.

[24]  Stan Z. Li,et al.  Extraction of feature subspaces for content-based retrieval using relevance feedback , 2001, MULTIMEDIA '01.

[25]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[26]  Yimin Wu,et al.  A feature re-weighting approach for relevance feedback in image retrieval , 2002, Proceedings. International Conference on Image Processing.

[27]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[28]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[29]  Thomas S. Huang,et al.  Comparing discriminating transformations and SVM for learning during multimedia retrieval , 2001, MULTIMEDIA '01.

[30]  Fabio Roli,et al.  Bayesian relevance feedback for content-based image retrieval , 2004, Pattern Recognit..

[31]  Thomas S. Huang,et al.  Content-based image retrieval with relevance feedback in MARS , 1997, Proceedings of International Conference on Image Processing.

[32]  Erkki Oja,et al.  Statistical Shape Features for Content-Based Image Retrieval , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[33]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[35]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[36]  John P. Oakley,et al.  Storage and Retrieval for Image and Video Databases , 1993 .

[37]  Ramin Zabih,et al.  Comparing images using color coherence vectors , 1997, MULTIMEDIA '96.

[38]  Igor Kononenko,et al.  On Biases in Estimating Multi-Valued Attributes , 1995, IJCAI.

[39]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .