Active Learning for Interactive Multimedia Retrieval

As the first decade of the 21st century comes to a close, growth in multimedia delivery infrastructure and public demand for applications built on this backbone are converging like never before. The push towards reaching truly interactive multimedia technologies becomes stronger as our media consumption paradigms continue to change. In this paper, we profile a technology leading the way in this revolution: active learning. Active learning is a strategy that helps alleviate challenges inherent in multimedia information retrieval through user interaction. We show how active learning is ideally suited for the multimedia information retrieval problem by giving an overview of the paradigm and component technologies used with special attention given to the application scenarios in which these technologies are useful. Finally, we give insight into the future of this growing field and how it fits into the larger context of multimedia information retrieval.

[1]  D. Angluin.  Queries and Concept Learning , 1988 .

[2]  Thomas S. Huang,et al.  Exploration of Visual Data , 2003, The Springer International Series in Video Computing.

[3]  Christos Faloutsos,et al.  MindReader: Querying Databases Through Multiple Examples , 1998, VLDB.

[4]  Thomas S. Huang,et al.  Relevance Feedback Techniques in Image Retrieval , 2001, Principles of Visual Information Retrieval.

[5]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[6]  Jonathan Foote,et al.  Content-based retrieval of music and audio , 1997, Other Conferences.

[7]  Paul A. Viola,et al.  Empirical Entropy Manipulation for Real-World Problems , 1995, NIPS.

[8]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[9]  E. T. Jaynes,et al.  BAYESIAN METHODS: GENERAL BACKGROUND ? An Introductory Tutorial , 1986 .

[10]  Thomas S. Huang,et al.  Evaluating group-based relevance feedback for content-based image retrieval , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[11]  Klaus Brinker,et al.  Active learning of label ranking functions , 2004, ICML.

[12]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[13]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[14]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[15]  Deok-Hwan Kim,et al.  QCluster: relevance feedback using adaptive clustering for content-based image retrieval , 2003, SIGMOD '03.

[16]  Yee Whye Teh,et al.  Names and faces in the news , 2004, CVPR 2004.

[17]  Suk I. Yoo,et al.  A Neural Network-Based Image Retrieval Using Nonlinear Combination of Heterogeneous Features , 2001, Int. J. Comput. Intell. Appl..

[18]  Daniel P. W. Ellis,et al.  Support vector machine active learning for music retrieval , 2006, Multimedia Systems.

[19]  John P. Eakins,et al.  Similarity Retrieval of Trademark Images , 1998, IEEE Multim..

[20]  Thomas S. Huang,et al.  Utilizing Information Theoretic Diversity for SVM Active Learn , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[21]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[22]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[23]  Bir Bhanu,et al.  Active concept learning for image retrieval in dynamic databases , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Thomas S. Huang,et al.  Combining diversity-based active learning with discriminant analysis in image retrieval , 2005, Third International Conference on Information Technology and Applications (ICITA'05).

[25]  Thomas S. Huang,et al.  Small sample learning during multimedia retrieval using BiasMap , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[26]  Marcel Worring,et al.  Goalgle: A Soccer Video Search Engine , 2003 .

[27]  Kriengkrai Porkaew,et al.  Query refinement for multimedia similarity retrieval in MARS , 1999, MULTIMEDIA '99.

[28]  David A. Forsyth,et al.  Words and Pictures in the News , 2003, HLT-NAACL 2003.

[29]  Kannan Ramchandran,et al.  Multimedia Analysis and Retrieval System (MARS) Project , 1996, Data Processing Clinic.

[30]  Arnold W. M. Smeulders,et al.  Everything Gets Better All the Time, Apart from the Amount of Data , 2004, CIVR.

[31]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[32]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[33]  Edward Y. Chang,et al.  Support vector machine active learning for image retrieval , 2001, MULTIMEDIA '01.

[34]  Thomas S. Huang,et al.  Leveraging Active Learning for Relevance Feedback Using an Information Theoretic Diversity Measure , 2006, CIVR.

[35]  Marcel Worring,et al.  Multimodal Video Indexing : A Review of the State-ofthe-art , 2001 .

[36]  Andrew McCallum,et al.  Toward Optimal Active Learning through Monte Carlo Estimation of Error Reduction , 2001, ICML 2001.

[37]  Edward Y. Chang,et al.  Multimodal concept-dependent active learning for image retrieval , 2004, MULTIMEDIA '04.

[38]  Niels Provos,et al.  Detecting Steganographic Content on the Internet , 2002, NDSS.

[39]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[40]  Shih-Fu Chang,et al.  Image Retrieval: Current Techniques, Promising Directions, and Open Issues , 1999, J. Vis. Commun. Image Represent..

[41]  Thomas S. Huang,et al.  A Discussion of Nonlinear Variants of Biased Discriminants for Interactive Image Retrieval , 2004, CIVR.

[42]  Meng Wang,et al.  Semi-automatic video annotation based on active learning with multiple complementary predictors , 2005, MIR '05.

[43]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.

[44]  Dan Roth,et al.  Generalization Bounds for the Area Under the ROC Curve , 2005, J. Mach. Learn. Res..

[45]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[46]  Michael Fink,et al.  Social- and Interactive-Television Applications Based on Real-Time Ambient-Audio Identification , 2006 .

[47]  Edward Y. Chang,et al.  Support Vector Machine Concept-Dependent Active Learning for Image Retrieval , 2005 .

[48]  ByoungChul Ko,et al.  Probabilistic neural networks supporting multi-class relevance feedback in region-based image retrieval , 2002, Object recognition supported by user interaction for service robots.

[49]  Daniel P. W. Ellis,et al.  Song-Level Features and Support Vector Machines for Music Classification , 2005, ISMIR.

[50]  Yanjun Qi,et al.  Automated analysis of nursing home observations , 2004, IEEE Pervasive Computing.

[51]  Jenq-Neng Hwang,et al.  Attentional focus training by boundary region data selection , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.