Learning mixed divergences in coupled matrix and tensor factorization models

Coupled tensor factorization methods are useful for sensor fusion, combining information from several related datasets by simultaneously approximating them by products of latent tensors. In these methods, the choice of a suitable optimization criteria becomes difficult as observed datasets may have different statistical characteristics and their relative importance for the task at hand can vary. In this paper, we present an algorithmic framework for coupled factorization that, while estimating a latent factorization also estimates a specific ß-divergence for each dataset as well as the relative weights in an overall additive cost function. We evaluate the proposed method on both synthetical and real datasets, where we apply our methods on a link prediction problem. The results show that our method outperforms the state-of-the-art by a significant margin.

[1]  R. A. van den Berg,et al.  Simultaneous analysis of coupled data matrices subject to different amounts of noise. , 2011, The British journal of mathematical and statistical psychology.

[2]  Erkki Oja,et al.  Selecting β-Divergence for Nonnegative Matrix Factorization by Score Matching , 2012, ICANN.

[3]  Ali Taylan Cemgil,et al.  Generalised Coupled Tensor Factorisation , 2011, NIPS.

[4]  Gordon K. Smyth,et al.  Series evaluation of Tweedie exponential dispersion model densities , 2005, Stat. Comput..

[5]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[6]  Minje Kim,et al.  Nonnegative Matrix Partial Co-Factorization for Spectral and Temporal Drum Source Separation , 2011, IEEE Journal of Selected Topics in Signal Processing.

[7]  Ananda Sen,et al.  The Theory of Dispersion Models , 1997, Technometrics.

[8]  Tom Barker,et al.  Ultrasound-coupled semi-supervised nonnegative matrix factorisation for speech enhancement , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  Ali Taylan Cemgil,et al.  Link prediction in heterogeneous data via generalized coupled tensor factorization , 2013, Data Mining and Knowledge Discovery.

[10]  Guangxi Zhu,et al.  On the testing for alpha-stable distributions of network traffic , 2004, Comput. Commun..

[11]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[12]  Yanwei Zhang,et al.  Likelihood-based and Bayesian methods for Tweedie compound Poisson linear mixed models , 2013, Stat. Comput..

[13]  Alexey Ozerov,et al.  Text-informed audio source separation using nonnegative matrix partial co-factorization , 2013, 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[14]  Ali Taylan Cemgil,et al.  Score guided musical source separation using Generalized Coupled Tensor Factorization , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[15]  Jacek B. Krawczyk Dependence of Left-Skewed Payoff Distributions on Risky-Asset Price Uncertainty , 2005 .

[16]  S. Godsill MCMC and EM-based methods for inference in heavy-tailed processes with /spl alpha/-stable innovations , 1999, Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics. SPW-HOS '99.

[17]  Tamara G. Kolda,et al.  All-at-once Optimization for Coupled Matrix and Tensor Factorizations , 2011, ArXiv.

[18]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[19]  Ali Taylan Cemgil,et al.  Learning the beta-Divergence in Tweedie Compound Poisson Matrix Factorization Models , 2013, ICML.

[20]  Xing Xie,et al.  Collaborative Filtering Meets Mobile Recommendation: A User-Centered Approach , 2010, AAAI.

[21]  Ali Taylan Cemgil,et al.  Optimal weight learning for Coupled Tensor Factorization with mixed divergences , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[22]  Rasmus Bro,et al.  Coupled Matrix Factorization with Sparse Factors to Identify Potential Biomarkers in Metabolomics , 2012, 2012 IEEE 12th International Conference on Data Mining Workshops.

[23]  R. Bro PARAFAC. Tutorial and applications , 1997 .

[24]  Erkki Oja,et al.  Learning the Information Divergence , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.