Multimodal concept-dependent active learning for image retrieval

It has been established that active learning is effective for learning complex, subjective query concepts for image retrieval. However, active learning has been applied in a concept independent way, (i.e., the kernel-parameters and the sampling strategy are identically chosen) for learning query concepts of differing <i>complexity</i>. In this work, we first characterize a concept's complexity using three measures: <i>hit-rate</i>, <i>isolation</i> and <i>diversity</i>. We then propose a multimodal learning approach that uses images' semantic labels to guide a <i>concept-dependent</i>, <i>active-learning</i> process. Based on the complexity of a concept, we make intelligent adjustments to the sampling strategy and the sampling pool from which images are to be selected and labeled, to improve concept learnability. Our empirical study on a $300$K-image dataset shows that concept-dependent learning is highly effective for image-retrieval accuracy.

[1]  Edward Y. Chang,et al.  On scalability of active learning for formulating query concepts , 2004, CVDB '04.

[2]  Lei Zhu,et al.  Theory of keyblock-based image retrieval , 2002, TOIS.

[3]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[5]  Edward Y. Chang,et al.  Clustering for Approximate Similarity Search in High-Dimensional Spaces , 2002, IEEE Trans. Knowl. Data Eng..

[6]  Mingjing Li,et al.  Relevance Feedback and Learning in Content-Based Image Search , 2004, World Wide Web.

[7]  Shih-Fu Chang,et al.  MediaNet: a multimedia information network for knowledge representation , 2000, SPIE Optics East.

[8]  Edward Y. Chang,et al.  MEGA---the maximizing expected generalization algorithm for learning complex query concepts , 2003, TOIS.

[9]  Thomas S. Huang,et al.  Unifying Keywords and Visual Contents in Image Retrieval , 2002, IEEE Multim..

[10]  Thijs Westerveld,et al.  Image Retrieval: Content versus Context , 2000, RIAO.

[11]  Tomasz Imielinski,et al.  Mining association rules between sets of items in large databases , 1993, SIGMOD Conference.

[12]  Qiang Yang,et al.  A unified framework for semantics and feature based relevance feedback in image retrieval systems , 2000, ACM Multimedia.

[13]  Shih-Fu Chang,et al.  Image classification using multimedia knowledge networks , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[14]  Kobus Barnard,et al.  Exploiting Text and Image Feature Co-occurrence Statistics in Large Datasets , 2003 .

[15]  ZhuLei,et al.  Theory of keyblock-based image retrieval , 2002 .

[16]  David A. Forsyth,et al.  Learning the semantics of words and pictures , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[18]  Klaus Brinker,et al.  Incorporating Diversity in Active Learning with Support Vector Machines , 2003, ICML.

[19]  Hironobu Takahashi,et al.  Automatic word assignment to images based on image division and vector quantization , 2000 .

[20]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[21]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  David G. Stork,et al.  Pattern Classification , 1973 .