Shift-Invariant Grouped Multi-task Learning for Gaussian Processes

Multi-task learning leverages shared information among data sets to improve the learning performance of individual tasks. The paper applies this framework for data where each task is a phase-shifted periodic time series. In particular, we develop a novel Bayesian nonparametric model capturing a mixture of Gaussian processes where each task is a sum of a group-specific function and a component capturing individual variation, in addition to each task being phase shifted. We develop an efficient em algorithm to learn the parameters of the model. As a special case we obtain the Gaussian mixture model and EM algorithm for phased-shifted periodic time series. Experiments in regression, classification and class discovery demonstrate the performance of the proposed model using both synthetic data and real-world time series data from astrophysics. Our methods are particularly useful when the time series are sparsely and non-synchronously sampled.

[1]  E. Stein,et al.  Real Analysis: Measure Theory, Integration, and Hilbert Spaces , 2005 .

[2]  Pavlos Protopapas,et al.  Eclipsing Binary Stars in the Large and Small Magellanic Clouds from the MACHO Project: The Sample , 2007, 0711.1617.

[3]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[4]  Marcin Kubiak,et al.  The Optical Gravitational Lensing Experiment , 1992 .

[5]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[6]  J. Sethuraman A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[7]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[8]  Emin Orhan Dirichlet Processes , 2012 .

[9]  Richard J. Povinelli,et al.  Time series classification using Gaussian mixture models of reconstructed phase spaces , 2004, IEEE Transactions on Knowledge and Data Engineering.

[10]  Volker Tresp,et al.  Mixtures of Gaussian Processes , 2000, NIPS.

[11]  Anton Schwaighofer,et al.  Learning Gaussian Process Kernels via Hierarchical Bayes , 2004, NIPS.

[12]  Christopher Bishop,et al.  Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics , 2003 .

[13]  Charles A. Micchelli,et al.  On Learning Vector-Valued Functions , 2005, Neural Computation.

[14]  Michael P Lesser,et al.  LSST Instrument Concept , 2002, SPIE Astronomical Telescopes + Instrumentation.

[15]  Lawrence Carin,et al.  Multi-Task Learning for Classification with Dirichlet Process Priors , 2007, J. Mach. Learn. Res..

[16]  Padhraic Smyth,et al.  Probabilistic curve-aligned clustering and prediction with regression mixture models , 2004 .

[17]  Padhraic Smyth,et al.  Segmental Hidden Markov Models with Random Effects for Waveform Modeling , 2006, J. Mach. Learn. Res..

[18]  R. Davies,et al.  Astronomical Society of the Pacific Conference Series , 2010 .

[19]  Michael I. Jordan,et al.  Variational inference for Dirichlet process mixtures , 2006 .

[20]  Carl E. Rasmussen,et al.  Infinite Mixtures of Gaussian Process Experts , 2001, NIPS.

[21]  Carl E. Rasmussen,et al.  Gaussian Processes for Machine Learning (GPML) Toolbox , 2010, J. Mach. Learn. Res..

[22]  Paolo Conconi,et al.  Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series , 2012 .

[23]  A U D A L S K I,et al.  Optical Gravitational Lensing Experiment. Photometry of the Macho-smc-1 Microlensing Candidate. * , 1997 .

[24]  Padhraic Smyth,et al.  Curve Clustering with Random Effects Regression Mixtures , 2003, AISTATS.

[25]  Padhraic Smyth,et al.  Translation-invariant mixture models for curve clustering , 2003, KDD '03.

[26]  Li Wei,et al.  Semi-supervised time series classification , 2006, KDD '06.

[27]  Walter A. Siegmund,et al.  Design of the Pan‐STARRS telescopes , 2004 .

[28]  Pavlos Protopapas,et al.  Kernels for Periodic Time Series Arising in Astronomy , 2009, ECML/PKDD.

[29]  Deniz Erdogmus,et al.  A reproducing kernel Hilbert space framework for pairwise time series distances , 2008, ICML '08.

[30]  P. Protopapas,et al.  Finding outlier light curves in catalogues of periodic variable stars , 2005, astro-ph/0505495.

[31]  Pavlos Protopapas,et al.  Finding anomalous periodic time series , 2009, Machine Learning.

[32]  Arnaud Doucet,et al.  Bayesian Unsupervised Signal Classification by Dirichlet Process Mixtures of Gaussian Processes , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[33]  Marcin Kubiak,et al.  The Optical Gravitational Lensing Experiment. Catalog of RR Lyr Stars in the Large Magellanic Cloud , 2003 .

[34]  Giuseppe De Nicolao,et al.  Bayesian Online Multitask Learning of Gaussian Processes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[36]  Charles A. Micchelli,et al.  Learning Multiple Tasks with Kernel Methods , 2005, J. Mach. Learn. Res..

[37]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[38]  Christopher W. Stubbs,et al.  The MACHO Project - a Search for the Dark Matter in the Milky-Way , 1993 .

[39]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[40]  Stanislaw Osowski,et al.  Support vector machine-based expert system for reliable heartbeat recognition , 2004, IEEE Transactions on Biomedical Engineering.

[41]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[42]  Roni Khardon,et al.  Kernel methods and their application to structured data , 2009 .

[43]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[44]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[45]  Giuseppe De Nicolao,et al.  Client–Server Multitask Learning From Distributed Datasets , 2008, IEEE Transactions on Neural Networks.

[46]  Thomas Lengauer,et al.  Multi-task learning for HIV therapy screening , 2008, ICML '08.

[47]  Padhraic Smyth,et al.  Joint Probabilistic Curve Clustering and Alignment , 2004, NIPS.

[48]  Murat Dundar,et al.  An Improved Multi-task Learning Approach with Applications in Medical Diagnosis , 2008, ECML/PKDD.

[49]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[50]  Anton Schwaighofer,et al.  Learning Gaussian processes from multiple tasks , 2005, ICML.

[51]  Stephen J. Roberts,et al.  Markov Models for Automated ECG Interval Analysis , 2003, NIPS.

[52]  Padhraic Smyth,et al.  Trajectory clustering with mixtures of regression models , 1999, KDD '99.