Bayesian Multi-view Tensor Factorization

We introduce a Bayesian extension of the tensor factorization problem to multiple coupled tensors. For a single tensor it reduces to standard PARAFAC-type Bayesian factorization, and for two tensors it is the first Bayesian Tensor Canonical Correlation Analysis method. It can also be seen to solve a tensorial extension of the recent Group Factor Analysis problem. The method decomposes the set of tensors to factors shared by subsets of the tensors, and factors private to individual tensors, and does not assume orthogonality. For a single tensor, the method empirically outperforms existing methods, and we demonstrate its performance on multiple tensor factorization tasks in toxicogenomics and functional neuroimaging.

[1]  J. Kruskal,et al.  A two-stage procedure incorporating good features of both trilinear and quadrilinear models , 1989 .

[2]  Rasmus Bro,et al.  The N-way Toolbox for MATLAB , 2000 .

[3]  Haiping Lu,et al.  Learning Canonical Correlations of Paired Tensor Sets Via Tensor-to-Vector Projection , 2013, IJCAI.

[4]  Ali Taylan Cemgil,et al.  Link prediction in heterogeneous data via generalized coupled tensor factorization , 2013, Data Mining and Knowledge Discovery.

[5]  Samuel Kaski,et al.  Bayesian Group Factor Analysis , 2012, AISTATS.

[6]  Xi Chen,et al.  Temporal Collaborative Filtering with Bayesian Probabilistic Tensor Factorization , 2010, SDM.

[7]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[8]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[9]  Xiaoyan Zhou,et al.  Sparse 2-D Canonical Correlation Analysis via Low Rank Matrix Approximation for Feature Extraction , 2012, IEEE Signal Processing Letters.

[10]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[11]  Pierre Comon,et al.  Canonical Polyadic Decomposition with a Columnwise Orthonormal Factor Matrix , 2012, SIAM J. Matrix Anal. Appl..

[12]  R. Cattell.  “Parallel proportional profiles” and other principles for determining the choice of factors by rotation , 1944 .

[13]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[14]  Wei Chu,et al.  Probabilistic Models for Incomplete Multi-dimensional Arrays , 2009, AISTATS.

[15]  Samuel Kaski,et al.  Bayesian Canonical correlation analysis , 2013, J. Mach. Learn. Res..

[16]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[17]  Paul A Clemons,et al.  The Connectivity Map: Using Gene-Expression Signatures to Connect Small Molecules, Genes, and Disease , 2006, Science.

[18]  R. Shoemaker.  The NCI60 human tumour cell line anticancer drug screen , 2006, Nature Reviews Cancer.

[19]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[20]  Seungjin Choi,et al.  Two-Dimensional Canonical Correlation Analysis , 2007, IEEE Signal Processing Letters.

[21]  L. Fritz,et al.  A high-affinity conformation of Hsp90 confers tumour selectivity on Hsp90 inhibitors , 2003, Nature.

[22]  Tae-Kyun Kim,et al.  Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Zenglin Xu,et al.  Infinite Tucker Decomposition: Nonparametric Bayesian Models for Multiway Data Analysis , 2011, ICML.

[24]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[25]  L. K. Hansen,et al.  Sparse Coding and Automatic Relevance Determination for Multiway models , 2009 .

[26]  Peter D. Hoff,et al.  Hierarchical multilinear models for multiway data , 2010, Comput. Stat. Data Anal..

[27]  Rasmus Bro,et al.  Understanding data fusion within the framework of coupled matrix and tensor factorizations , 2013 .

[28]  A. Stegeman,et al.  On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC Model , 2008, Psychometrika.

[29]  Kazushi Ikeda,et al.  Exponential family tensor factorization: an online extension and applications , 2012, Knowledge and Information Systems.

[30]  Thomas Hartung,et al.  Food for Thought … Systems Toxicology , 2012 .

[31]  Ali Taylan Cemgil,et al.  Generalised Coupled Tensor Factorisation , 2011, NIPS.

[32]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .