A framework for classifier adaptation and its applications in concept detection

There is often a need to adapt supervised classifiers such as semantic concept detectors across different domains of data. This paper describes a generic framework for function-level classifier adaptation based on regularized loss minimization. It directly modifies the decision function of an existing classifier of any type into a classifier for a new domain, based on limited labeled data in the new domain and no "old data", which makes it an efficient and flexible framework. We then extend this framework to adapt multiple classifiers into one classifier, with the weights of existing classifiers learned automatically to reflect their utility. We elaborate on two concrete adaptation algorithms derived from the framework, namely adaptive SVM and multi-adaptive SVM, for one-to-one and many-to-one adaptation respectively. In the experiments of adapting semantic concept detectors across video channels/types, our adaptation approach is proven to be superior to using original (unadapted) classifiers or building new ones in terms of accuracy and labeling effort.

[1]  Thorsten Joachims,et al.  Detecting Concept Drift with Support Vector Machines , 2000, ICML.

[2]  Tao Mei,et al.  Correlative multi-label video annotation , 2007, ACM Multimedia.

[3]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[4]  John R. Smith,et al.  Semantic representation: search and mining of multimedia content , 2004, KDD '04.

[5]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[6]  Thomas G. Dietterich,et al.  Improving SVM accuracy by training on auxiliary data sources , 2004, ICML.

[7]  Mads Haahr,et al.  A Case-Based Approach to Spam Filtering that Can Track Concept Drift , 2003 .

[8]  Brian Roark,et al.  Supervised and unsupervised PCFG adaptation to novel domains , 2003, NAACL.

[9]  Brian Roark,et al.  Unsupervised language model adaptation , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[10]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[11]  Rajat Raina,et al.  Constructing informative priors using transfer learning , 2006, ICML.

[12]  Jun Yang,et al.  (Un)Reliability of video concept detection , 2008, CIVR '08.

[13]  Sebastian Thrun,et al.  Is Learning The n-th Thing Any Easier Than Learning The First? , 1995, NIPS.

[14]  Rong Yan,et al.  Cross-domain video concept detection using adaptive svms , 2007, ACM Multimedia.

[15]  Lawrence Carin,et al.  Logistic regression with an auxiliary data source , 2005, ICML.

[16]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[17]  Thomas G. Dietterich,et al.  Transfer Learning with an Ensemble of Background Tasks , 2005, NIPS 2005.

[18]  John R. Smith,et al.  IBM Research TRECVID-2009 Video Retrieval System , 2009, TRECVID.