Learning kernel combination from noisy pairwise constraints

We consider the problem of learning the combination of multiple kernels given noisy pairwise constraints, which is in contrast to most of the existing studies that assume perfect pairwise constraints. This problem is particularly important when the pairwise constraints are derived from side information such as hyperlinks and paper citations. We propose a probabilistic approach for learning the combination of multiple kernels and show that under appropriate assumptions, the combination weights learned by the proposed approach from the noisy pairwise constraints converge to the optimal weights learned from perfectly labeled pairwise constraints. Empirical studies on data clustering using the learned combined kernel verify the effectiveness of the proposed approach.

[1]  Rong Jin,et al.  Multi-label Multiple Kernel Learning by Stochastic Approximation: Application to Visual Object Recognition , 2010, NIPS.

[2]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[3]  Gideon S. Mann,et al.  Putting Semantic Information Extraction on the Map : Noisy Label Models for Fact Extraction , 2007 .

[4]  Alexander J. Smola,et al.  Estimating Labels from Label Proportions , 2009, J. Mach. Learn. Res..

[5]  Pushpak Bhattacharyya,et al.  A model for handling approximate, noisy or incomplete labeling in text classification , 2005, ICML.

[6]  Yun Chi,et al.  Combining link and content for community detection: a discriminative approach , 2009, KDD.

[7]  Rong Jin,et al.  Learning from Noisy Side Information by Generalized Maximum Entropy Model , 2010, ICML.

[8]  Ethem Alpaydin,et al.  Localized multiple kernel learning , 2008, ICML '08.

[9]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[10]  Rong Jin,et al.  Learning nonparametric kernel matrices from pairwise constraints , 2007, ICML '07.

[11]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[12]  Raymond J. Mooney,et al.  A probabilistic framework for semi-supervised clustering , 2004, KDD.

[13]  Ethem Alpaydin,et al.  Localized algorithms for multiple kernel learning , 2013, Pattern Recognit..

[14]  Edward Y. Chang,et al.  Learning the unified kernel machines for classification , 2006, KDD '06.