Multi-set Canonical Correlation Analysis simply explained.

There are a multitude of methods to perform multi-set correlated component analysis (MCCA), including some that require iterative solutions. The methods differ on the criterion they optimize and the constraints placed on the solutions. This note focuses perhaps on the simplest version, which can be solved in a single step as the eigenvectors of matrix ${\bf D}^{-1} {\bf R}$. Here ${\bf R}$ is the covariance matrix of the concatenated data, and ${\bf D}$ is its block-diagonal. This note shows that this solution maximizes inter-set correlation (ISC) without further constraints. It also relates the solution to a two step procedure, which first whitens each dataset using PCA, and then performs an additional PCA on the concatenated and whitened data. Both these solutions are known, although a clear derivation and simple implementation are hard to find. This short note aims to remedy this.

[1]  Nicholas Asendorf,et al.  Informative Data Fusion: Beyond Canonical Correlation Analysis , 2015 .

[2]  Lars Kai Hansen,et al.  Multiview Bayesian Correlated Component Analysis , 2015, Neural Computation.

[3]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[4]  Lucas C. Parra,et al.  Correlated Components Analysis - Extracting Reliable Dimensions in Multivariate Data , 2018, Neurons, Behavior, Data analysis, and Theory.

[5]  Chong-sun Kim Canonical Analysis of Several Sets of Variables , 1973 .

[6]  Paul Horst,et al.  Relations amongm sets of measures , 1961 .

[7]  William Meredith,et al.  Rotation to achieve factorial invariance , 1964 .

[8]  P. Horst Generalized canonical correlations and their applications to experimental data. , 1961, Journal of clinical psychology.

[9]  Nikos D. Sidiropoulos,et al.  Scalable and Flexible Multiview MAX-VAR Canonical Correlation Analysis , 2016, IEEE Transactions on Signal Processing.

[10]  J. Gower Generalized procrustes analysis , 1975 .

[11]  Allan Aasbjerg Nielsen,et al.  Multiset canonical correlations analysis and multispectral, truly multitemporal remote sensing data , 2002, IEEE Trans. Image Process..

[12]  John C. Gower,et al.  A synthesis of canonical variate analysis, generalised canonical correlation and Procrustes analysis , 2006, Comput. Stat. Data Anal..

[13]  Hao Xu,et al.  Regularized hyperalignment of multi-set fMRI data , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[14]  Bryan R. Conroy,et al.  A Common, High-Dimensional Model of the Representational Space in Human Ventral Temporal Cortex , 2011, Neuron.