Robust multi-task learning with t-processes

Most current multi-task learning frameworks ignore the robustness issue, which means that the presence of "outlier" tasks may greatly reduce overall system performance. We introduce a robust framework for Bayesian multitask learning, t-processes (TP), which are a generalization of Gaussian processes (GP) for multi-task learning. TP allows the system to effectively distinguish good tasks from noisy or outlier tasks. Experiments show that TP not only improves overall system performance, but can also serve as an indicator for the "informativeness" of different tasks.

[1]  Yiming Yang,et al.  Learning Multiple Related Tasks using Latent Independent Component Analysis , 2005, NIPS.

[2]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[3]  Tong Zhang,et al.  A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , 2005, J. Mach. Learn. Res..

[4]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[5]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[6]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[7]  M. Steel,et al.  Multivariate Student -t Regression Models : Pitfalls and Inference , 1999 .

[8]  Jerry Nedelman,et al.  Book review: “Bayesian Data Analysis,” Second Edition by A. Gelman, J.B. Carlin, H.S. Stern, and D.B. Rubin Chapman & Hall/CRC, 2004 , 2005, Comput. Stat..

[9]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[10]  D. Rubin,et al.  ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME , 1999 .

[11]  Samuel Kotz,et al.  Multivariate T-Distributions and Their Applications , 2004 .

[12]  Anton Schwaighofer,et al.  Hierarchical Bayesian modelling with Gaus-sian processes , 2005, NIPS 2005.

[13]  Anton Schwaighofer,et al.  Learning Gaussian processes from multiple tasks , 2005, ICML.

[14]  Tom Heskes,et al.  Task Clustering and Gating for Bayesian Multitask Learning , 2003, J. Mach. Learn. Res..

[15]  Hans-Peter Kriegel,et al.  Collaborative ordinal regression , 2006, ICML.

[16]  Saralees Nadarajah,et al.  Multivariate T-Distributions and Their Applications , 2004 .

[17]  Rich Caruana,et al.  Multitask Learning , 1997, Machine-mediated learning.

[18]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.