Linked Component Analysis From Matrices to High-Order Tensors: Applications to Biomedical Data

With the increasing availability of various sensor technologies, we now have access to large amounts of multiblock (also called multiset, multirelational, or multiview) data that need to be jointly analyzed to explore their latent connections. Various component analysis methods have played an increasingly important role for the analysis of such coupled data. In this article, we first provide a brief review of existing matrix-based (two-way) component analysis methods for the joint analysis of such data with a focus on biomedical applications. Then, we discuss their important extensions and generalization to multiblock multiway (tensor) data. We show how constrained multiblock tensor decomposition methods are able to extract similar or statistically dependent common features that are shared by all blocks, by incorporating the multiway nature of data. Special emphasis is given to the flexible common and individual feature analysis of multiblock data with the aim to simultaneously extract common and individual latent components with desired properties and types of diversity. Illustrative examples are given to demonstrate their effectiveness for biomedical data analysis.

[1]  Donald Goldfarb,et al.  Robust Low-Rank Tensor Recovery: Models and Algorithms , 2013, SIAM J. Matrix Anal. Appl..

[2]  G. Golub,et al.  A tensor higher-order singular value decomposition for integrative analysis of DNA microarray data from different studies , 2007, Proceedings of the National Academy of Sciences.

[3]  Eric F Lock,et al.  JOINT AND INDIVIDUAL VARIATION EXPLAINED (JIVE) FOR INTEGRATED ANALYSIS OF MULTIPLE DATA TYPES. , 2011, The annals of applied statistics.

[4]  H. Hotelling Relations Between Two Sets of Variates , 1936 .

[5]  Vince D. Calhoun,et al.  Independent Vector Analysis for Gradient Artifact Removal in Concurrent EEG-fMRI Data , 2015, IEEE Transactions on Biomedical Engineering.

[6]  Wei Wu,et al.  Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs , 2007, IEEE Transactions on Biomedical Engineering.

[7]  Christian Jutten,et al.  Joint blind source separation of multidimensional components: Model and algorithm , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[8]  T. Adali,et al.  Unmixing fMRI with independent component analysis , 2006, IEEE Engineering in Medicine and Biology Magazine.

[9]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[10]  Michael W. Mahoney Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[11]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[12]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[13]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[14]  Xiaofeng Gong,et al.  Tensor decomposition of EEG signals: A brief review , 2015, Journal of Neuroscience Methods.

[15]  Andrzej Cichocki,et al.  Fast and unique Tucker decompositions via multiway blind source separation , 2012 .

[16]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Andreas Ziehe,et al.  A Fast Algorithm for Joint Diagonalization with Non-orthogonal Transformations and its Application to Blind Source Separation , 2004, J. Mach. Learn. Res..

[18]  Nikos D. Sidiropoulos,et al.  Turbo-SMT: Accelerating Coupled Sparse Matrix-Tensor Factorizations by 200x , 2014, SDM.

[19]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[20]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[21]  Liqing Zhang,et al.  Natural gradient algorithm for blind separation of overdetermined mixture with additive noise , 1999, IEEE Signal Processing Letters.

[22]  Lars Kai Hansen,et al.  ERPWAVELAB A toolbox for multi-channel analysis of time–frequency transformed event related potentials , 2007, Journal of Neuroscience Methods.

[23]  Zenglin Xu,et al.  Infinite Tucker Decomposition: Nonparametric Bayesian Models for Multiway Data Analysis , 2011, ICML.

[24]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[25]  Guillermo Sapiro,et al.  Dimensionality Reduction via Subspace and Submanifold Learning [From the Guest Editors] , 2011, IEEE Signal Process. Mag..

[26]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[27]  Pierre Comon,et al.  Blind source separation of underdetermined mixtures of event-related sources , 2014, Signal Process..

[28]  Andrzej Cichocki,et al.  L1-Regularized Multiway Canonical Correlation Analysis for SSVEP-Based BCI , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[30]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[31]  Saeid Sanei,et al.  Artifact removal from electroencephalograms using a hybrid BSS-SVM algorithm , 2005, IEEE Signal Processing Letters.

[32]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorizations : An algorithmic perspective , 2014, IEEE Signal Processing Magazine.

[33]  Jin Tang,et al.  Graph-Laplacian PCA: Closed-Form Solution and Robustness , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Andrzej Cichocki,et al.  Era of Big Data Processing: A New Approach via Tensor Networks and Tensor Decompositions , 2014, ArXiv.

[35]  Roman Orus,et al.  A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States , 2013, 1306.2164.

[36]  Chong-Yung Chi,et al.  Nonnegative Least-Correlated Component Analysis for Separation of Dependent Sources by Volume Maximization , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[38]  Vince D. Calhoun,et al.  Multimodal Data Fusion Using Source Separation: Two Effective Models Based on ICA and IVA and Their Properties , 2015, Proceedings of the IEEE.

[39]  Vince D. Calhoun,et al.  Preserving subject variability in group fMRI analysis: performance evaluation of GICA vs. IVA , 2014, Front. Syst. Neurosci..

[40]  Andrzej Cichocki,et al.  Common components analysis via linked blind source separation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[41]  F Wendling,et al.  EEG extended source localization: Tensor-based vs. conventional methods , 2014, NeuroImage.

[42]  Volkan Cevher,et al.  Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics , 2014, IEEE Signal Processing Magazine.

[43]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[44]  Trac D. Tran,et al.  Tensor sparsification via a bound on the spectral norm of random tensors , 2010, ArXiv.

[45]  Xingyu Wang,et al.  Frequency Recognition in SSVEP-Based BCI using Multiset Canonical Correlation Analysis , 2013, Int. J. Neural Syst..

[46]  C. F. Beckmann,et al.  Tensorial extensions of independent component analysis for multisubject FMRI analysis , 2005, NeuroImage.

[47]  Saeid Sanei,et al.  EEG signal processing , 2000, Clinical Neurophysiology.

[48]  Gaojie Chen,et al.  Independent vector analysis with a generalized multivariate Gaussian source prior for frequency domain blind source separation , 2014, Signal Process..

[49]  Vince D. Calhoun,et al.  Neuronal chronometry of target detection: Fusion of hemodynamic and event-related potential data , 2005, NeuroImage.

[50]  Rasmus Bro,et al.  Coupled Matrix Factorization with Sparse Factors to Identify Potential Biomarkers in Metabolomics , 2012, 2012 IEEE 12th International Conference on Data Mining Workshops.

[51]  Hong Cheng,et al.  Generalized Higher-Order Orthogonal Iteration for Tensor Decomposition and Completion , 2014, NIPS.

[52]  Hualiang Li,et al.  Complex ICA Using Nonlinear Functions , 2008, IEEE Transactions on Signal Processing.

[53]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Rasmus Bro,et al.  Understanding data fusion within the framework of coupled matrix and tensor factorizations , 2013 .

[55]  Nitish V. Thakor IEEE Transactions on Neural Systems and Rehabilitation Engineering: Editorial , 2006 .

[56]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[57]  V D Calhoun,et al.  Spatial and temporal independent component analysis of functional MRI data containing a pair of task‐related waveforms , 2001, Human brain mapping.

[58]  Morten Mørup,et al.  Non-negative Tensor Factorization with missing data for the modeling of gene expressions in the Human Brain , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[59]  Vince D. Calhoun,et al.  Joint Blind Source Separation by Multiset Canonical Correlation Analysis , 2009, IEEE Transactions on Signal Processing.

[60]  Guillermo Sapiro,et al.  Dimensionality Reduction via Subspace and Submanifold Learning , 2011 .

[61]  Vince D. Calhoun,et al.  Canonical Correlation Analysis for Data Fusion and Group Inferences , 2010, IEEE Signal Processing Magazine.

[62]  Andrzej Cichocki,et al.  Equivariant Nonstationary Source Separation , 2002 .

[63]  Nicolas Gillis,et al.  Fast and Robust Recursive Algorithmsfor Separable Nonnegative Matrix Factorization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Anand Rangarajan,et al.  Image Denoising Using the Higher Order Singular Value Decomposition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  W. Hackbusch Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[66]  Zhaoshui He,et al.  Minimum-Volume-Constrained Nonnegative Matrix Factorization: Enhanced Ability of Learning Parts , 2011, IEEE Transactions on Neural Networks.

[67]  Vince D. Calhoun,et al.  Capturing group variability using IVA: A simulation study and graph-theoretical analysis , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[68]  Kyuwan Choi,et al.  Detecting the Number of Clusters in n-Way Probabilistic Clustering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Saeid Sanei,et al.  EEG-FMRI integration using a partially constrained tensor factorization , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[70]  Giorgio Ottaviani,et al.  An Algorithm For Generic and Low-Rank Specific Identifiability of Complex Tensors , 2014, SIAM J. Matrix Anal. Appl..

[71]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[72]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[73]  Mark W. Woolrich,et al.  Linked independent component analysis for multimodal data fusion , 2011, NeuroImage.

[74]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[75]  Ying Guo,et al.  A unified framework for group independent component analysis for multi-subject fMRI data , 2008, NeuroImage.

[76]  Florian Roemer,et al.  Multi-dimensional space-time-frequency component analysis of event related EEG data using closed-form PARAFAC , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[77]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[78]  Andrzej Cichocki,et al.  Fast Nonnegative Matrix/Tensor Factorization Based on Low-Rank Approximation , 2012, IEEE Transactions on Signal Processing.

[79]  Babak Hossein Khalaj,et al.  A unified approach to sparse signal processing , 2009, EURASIP Journal on Advances in Signal Processing.

[80]  Ronald Phlypo,et al.  Independent Vector Analysis: Identification Conditions and Performance Bounds , 2013, IEEE Transactions on Signal Processing.

[81]  Liqing Zhang,et al.  Kernelization of Tensor-Based Models for Multiway Data Analysis: Processing of Multidimensional Structured Data , 2013, IEEE Signal Processing Magazine.

[82]  Andrzej Cichocki,et al.  Two Efficient Algorithms for Approximately Orthogonal Nonnegative Matrix Factorization , 2015, IEEE Signal Processing Letters.

[83]  Xavier Bresson,et al.  Robust Principal Component Analysis on Graphs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[84]  Vince D. Calhoun,et al.  A study of spatial variation in fMRI brain networks via independent vector analysis: Application to schizophrenia , 2014, 2014 International Workshop on Pattern Recognition in Neuroimaging.

[85]  Jason Farquhar,et al.  Shared processing of perception and imagery of music in decomposed EEG , 2013, NeuroImage.

[86]  Yang Song,et al.  Determining the number of correlated signals between two data sets using PCA-CCA when sample support is extremely small , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[87]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[88]  Tülay Adalı,et al.  Diversity in Independent Component and Vector Analyses: Identifiability, algorithms, and applications in medical imaging , 2014, IEEE Signal Processing Magazine.

[89]  Andrzej Cichocki,et al.  Smooth PARAFAC Decomposition for Tensor Completion , 2015, IEEE Transactions on Signal Processing.

[90]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[91]  Sungjin Hong,et al.  A critique of Tensor Probabilistic Independent Component Analysis: Implications and recommendations for multi-subject fMRI data analysis , 2013, Journal of Neuroscience Methods.

[92]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[93]  Shengli Xie,et al.  Mixing Matrix Estimation From Sparse Mixtures With Unknown Number of Sources , 2011, IEEE Transactions on Neural Networks.

[94]  Lieven De Lathauwer,et al.  Fourth-Order Cumulant-Based Blind Identification of Underdetermined Mixtures , 2007, IEEE Transactions on Signal Processing.

[95]  Andrzej Cichocki,et al.  Tensor Networks for Big Data Analytics and Large-Scale Optimization Problems , 2014, ArXiv.

[96]  Andrzej Cichocki,et al.  Accelerated Canonical Polyadic Decomposition Using Mode Reduction , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[97]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[98]  David B. Dunson,et al.  Scalable Bayesian Low-Rank Decomposition of Incomplete Multiway Tensors , 2014, ICML.

[99]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[100]  J. Kettenring,et al.  Canonical Analysis of Several Sets of Variables , 2022 .

[101]  Demetri Terzopoulos,et al.  Multilinear independent components analysis , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[102]  Andrzej Cichocki,et al.  Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[103]  Andrzej Cichocki,et al.  Efficient Nonnegative Tucker Decompositions: Algorithms and Uniqueness , 2014, IEEE Transactions on Image Processing.

[104]  Marko Filipovic,et al.  Tucker factorization with missing data with application to low-$$n$$n-rank tensor completion , 2015, Multidimens. Syst. Signal Process..

[105]  Wim Van Paesschen,et al.  Block term decomposition for modelling epileptic seizures , 2014, EURASIP J. Adv. Signal Process..

[106]  Pierre Comon,et al.  Uniqueness of Nonnegative Tensor Approximations , 2014, IEEE Transactions on Information Theory.

[107]  Liqing Zhang,et al.  Bayesian Robust Tensor Factorization for Incomplete Multiway Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[108]  Wim Van Paesschen,et al.  Canonical Decomposition of Ictal Scalp EEG and Accurate Source Localisation: Principles and Simulation Study , 2007, Comput. Intell. Neurosci..

[109]  Yukihiko Yamashita,et al.  Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis , 2015, Signal Process..

[110]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[111]  J. Pekar,et al.  A method for making group inferences from functional MRI data using independent component analysis , 2001, Human brain mapping.

[112]  Vince D. Calhoun,et al.  A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data , 2009, NeuroImage.

[113]  P. Kroonenberg Applied Multiway Data Analysis , 2008 .

[114]  Andrzej Cichocki,et al.  Canonical Polyadic Decomposition Based on a Single Mode Blind Source Separation , 2012, IEEE Signal Processing Letters.

[115]  Jonathon A. Chambers,et al.  Fetal electrocardiogram extraction by sequential source separation in the wavelet domain , 2005, IEEE Transactions on Biomedical Engineering.

[116]  Lieven De Lathauwer,et al.  Optimization-Based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-(Lr, Lr, 1) Terms, and a New Generalization , 2013, SIAM J. Optim..

[117]  Naotaka Fujii,et al.  Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[118]  Sabine Van Huffel,et al.  A Combination of Parallel Factor and Independent Component Analysis , 2022 .

[119]  Jun Zhang,et al.  Nonorthogonal Approximate Joint Diagonalization With Well-Conditioned Diagonalizers , 2009, IEEE Transactions on Neural Networks.

[120]  A. Cichocki,et al.  Generalizing the column–row matrix decomposition to multi-way arrays , 2010 .

[121]  Liqing Zhang,et al.  Bayesian Sparse Tucker Models for Dimension Reduction and Tensor Completion , 2015, ArXiv.

[122]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[123]  V. Calhoun,et al.  Multisubject Independent Component Analysis of fMRI: A Decade of Intrinsic Networks, Default Mode, and Neurodiagnostic Discovery , 2012, IEEE Reviews in Biomedical Engineering.

[124]  Nicolas Gillis,et al.  Sparse and unique nonnegative matrix factorization through data preprocessing , 2012, J. Mach. Learn. Res..

[125]  François-Benoît Vialatte,et al.  Independent vector analysis for SSVEP signal enhancement , 2015, 2015 49th Annual Conference on Information Sciences and Systems (CISS).

[126]  Laurent Albera,et al.  Fourth-order blind identification of underdetermined mixtures of sources (FOBIUM) , 2005, IEEE Transactions on Signal Processing.

[127]  Xinyuan Zhang,et al.  Denoising of 3D magnetic resonance images by using higher-order singular value decomposition , 2015, Medical Image Anal..

[128]  Vince D. Calhoun,et al.  Multidataset independent subspace analysis extends independent vector analysis , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[129]  Tülay Adalı,et al.  Capturing subject variability in fMRI data: A graph-theoretical analysis of GICA vs. IVA , 2015, Journal of Neuroscience Methods.

[130]  Sheng Luo,et al.  Population Value Decomposition, a Framework for the Analysis of Image Populations , 2011, Journal of the American Statistical Association.