Variational Bayesian Mixture of Robust CCA Models

We study the problem of extracting statistical dependencies between multivariate signals, to be used for exploratory analysis of complicated natural phenomena. In particular, we develop generative models for extracting the dependencies, made possible by the probabilistic interpretation of canonical correlation analysis (CCA). We introduce a mixture of robust canonical correlation analyzers, using t-distribution to make the model robust to outliers and variational Bayesian inference for learning from noisy data. We demonstrate the improvements of the new model on artificial data, and further apply it for analyzing dependencies between MEG and measurements of autonomic nervous system to illustrate potential use scenarios.

[1]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[2]  D. Rubin,et al.  ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME , 1999 .

[3]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[4]  Matthew J. Beal Variational algorithms for approximate Bayesian inference , 2003 .

[5]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[6]  Carla E. Brodley,et al.  Correlation Clustering for Learning Mixtures of Canonical Correlation Models , 2005, SDM.

[7]  Michael I. Jordan,et al.  A Probabilistic Interpretation of Canonical Correlation Analysis , 2005 .

[8]  Michel Verleysen,et al.  Robust probabilistic projections , 2006, ICML.

[9]  Michel Verleysen,et al.  Robust Bayesian clustering , 2007, Neural Networks.

[10]  K. Onoda,et al.  Anticipation of affective images and event-related desynchronization (ERD) of alpha activity: An MEG study , 2007, Brain Research.

[11]  Chong Wang,et al.  Variational Bayesian Approach to Canonical Correlation Analysis , 2007, IEEE Transactions on Neural Networks.

[12]  Samuel Kaski,et al.  Local dependent components , 2007, ICML '07.

[13]  Vince D. Calhoun,et al.  Canonical Correlation Analysis for Feature-Based Fusion of Biomedical Imaging Modalities and Its Application to Detection of Associative Networks in Schizophrenia , 2008, IEEE Journal of Selected Topics in Signal Processing.

[14]  Junbin Gao,et al.  Robust L1 Principal Component Analysis and Its Bayesian Variational Inference , 2008, Neural Computation.

[15]  Sotirios Chatzis,et al.  Signal Modeling and Classification Using a Robust Latent Space Model Based on $t$ Distributions , 2008, IEEE Transactions on Signal Processing.

[16]  Juha Karhunen,et al.  Bayesian Robust PCA of Incomplete Data , 2009, Neural Processing Letters.

[17]  Samuel Kaski,et al.  Dependencies between stimuli and spatially independent fMRI sources: Towards brain correlates of natural stimuli , 2009, NeuroImage.