Independent component analysis for tensor-valued data

Abstract In preprocessing tensor-valued data, e.g., images and videos, a common procedure is to vectorize the observations and subject the resulting vectors to one of the many methods used for independent component analysis (ICA). However, the tensor structure of the original data is lost in the vectorization and, as a more suitable alternative, we propose the matrix- and tensor fourth order blind identification (MFOBI and TFOBI). In these tensorial extensions of the classic fourth order blind identification (FOBI) we assume a Kronecker structure for the mixing and perform FOBI simultaneously on each direction of the observed tensors. We discuss the theory and assumptions behind MFOBI and TFOBI and provide two different algorithms and related estimates of the unmixing matrices along with their asymptotic properties. Finally, simulations are used to compare the method’s performance with that of classical FOBI for vectorized data and we end with a real data clustering example.

[1]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[2]  Su-Yun Huang,et al.  On multilinear principal component analysis of order-two tensors , 2011, 1104.5281.

[3]  Haiping Lu,et al.  A survey of multilinear subspace learning for tensor data , 2011, Pattern Recognit..

[4]  Demetri Terzopoulos,et al.  Multilinear independent components analysis , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[6]  Pauliina Ilmonen,et al.  Characteristics of multivariate distributions and the invariant coordinate system , 2010 .

[7]  James R. Schott,et al.  Tests for Kronecker envelope models in multilinear principal components analysis , 2014 .

[8]  Hongtu Zhu,et al.  Tensor Regression with Applications in Neuroimaging Data Analysis , 2012, Journal of the American Statistical Association.

[9]  R. Dennis Cook,et al.  Tensor sliced inverse regression , 2015, J. Multivar. Anal..

[10]  Visa Koivunen,et al.  Robust and sparse estimation of tensor decompositions , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[11]  Lexin Li,et al.  Regularized matrix regression , 2012, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[12]  Chris A. J. Klaassen,et al.  Squared skewness minus kurtosis bounded by 186/125 for unimodal distributions , 2000 .

[13]  Joni Virta,et al.  JADE for Tensor-Valued Observations , 2016, Journal of Computational and Graphical Statistics.

[14]  David E. Tyler,et al.  Invariant co‐ordinate selection , 2009 .

[15]  Brian D. Ripley,et al.  Modern Applied Statistics with S Fourth edition , 2002 .

[16]  Joni Virta,et al.  Blind source separation of tensor-valued time series , 2017, Signal Process..

[17]  K. Nordhausen,et al.  Joint Use of Third and Fourth Cumulants in Independent Component Analysis , 2015, 1505.02613.

[18]  Alfred O. Hero,et al.  Robust Kronecker Product PCA for Spatio-Temporal Covariance Estimation , 2015, IEEE Transactions on Signal Processing.

[19]  Joni Virta,et al.  Asymptotic and Bootstrap Tests for the Dimension of the Non-Gaussian Subspace , 2017, IEEE Signal Processing Letters.

[20]  A. Rukhin Matrix Variate Distributions , 1999, The Multivariate Normal Distribution.

[21]  David Zhang,et al.  Directional independent component analysis with tensor representation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  R. Cook,et al.  Dimension folding PCA and PFC for matrix-valued predictors , 2014 .

[23]  Fetsje Bijma,et al.  Existence and uniqueness of the maximum likelihood estimator for models with a Kronecker product covariance structure , 2014, J. Multivar. Anal..

[24]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[25]  T. Rao,et al.  Tensor Methods in Statistics , 1989 .

[26]  Dietrich von Rosen,et al.  The multilinear normal distribution: Introduction and some basic properties , 2013, J. Multivar. Anal..

[27]  R. Cook,et al.  Higher‐order sliced inverse regressions , 2015 .

[28]  L. Forzani,et al.  Sufficient dimension reduction for longitudinally measured predictors , 2012, Statistics in medicine.

[29]  Klaus Nordhausen,et al.  Deflation-Based FastICA With Adaptive Choices of Nonlinearities , 2014, IEEE Transactions on Signal Processing.

[30]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[31]  M. Srivastava,et al.  Models with a Kronecker product covariance structure: Estimation and testing , 2008 .

[32]  K. Nordhausen,et al.  Fourth Moments and Independent Component Analysis , 2014, 1406.4765.

[33]  Klaus Nordhausen,et al.  Tools for Exploring Multivariate Data: The Package ICS , 2008 .

[34]  Francisco J. Prieto,et al.  Eigenvectors of a kurtosis matrix as interesting directions to reveal cluster structure , 2010, J. Multivar. Anal..

[35]  Chenlei Leng,et al.  STRUCTURED LASSO FOR REGRESSION WITH MATRIX COVARIATES , 2014 .

[36]  Klaus Nordhausen,et al.  Blind Source Separation Based on Joint Diagonalization in R: The Packages JADE and BSSasymp , 2017 .

[37]  C. F. Beckmann,et al.  Tensorial extensions of independent component analysis for multisubject FMRI analysis , 2005, NeuroImage.

[38]  Wenxuan Zhong,et al.  Dimension Reduction for Tensor Classification , 2013 .

[39]  Ami Wiesel,et al.  Geodesic Convexity and Covariance Estimation , 2012, IEEE Transactions on Signal Processing.

[40]  Petre Stoica,et al.  On Estimation of Covariance Matrices With Kronecker Product Structure , 2008, IEEE Transactions on Signal Processing.

[41]  Prabhu Babu,et al.  Robust estimation of structured covariance matrix for heavy-tailed distributions , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[43]  Klaus Nordhausen,et al.  Asymptotic and bootstrap tests for subspace dimension , 2016, J. Multivar. Anal..

[44]  N. Altman,et al.  On dimension folding of matrix- or array-valued statistical objects , 2010, 1002.4789.

[45]  Steven Roman Advanced Linear Algebra , 1992 .

[46]  Hung Hung,et al.  Matrix variate logistic regression model with application to EEG data. , 2011, Biostatistics.

[47]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[48]  K. Nordhausen,et al.  Applying fully tensorial ICA to fMRI data , 2016, 2016 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[49]  Xiangrong Yin,et al.  Sufficient Dimension Folding for Regression Mean Function , 2014 .

[50]  V. Pan,et al.  The Complexity of the AlgebraicEigenproblem , 1998 .

[51]  Klaus Nordhausen,et al.  A New Performance Index for ICA: Properties, Computation and Asymptotic Analysis , 2010, LVA/ICA.

[52]  Jean-Francois Cardoso,et al.  Source separation using higher order moments , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[53]  Xin Xing,et al.  Tensor sufficient dimension reduction , 2015, Wiley interdisciplinary reviews. Computational statistics.

[54]  Pierre Dutilleul,et al.  Maximum likelihood estimation for the tensor normal distribution: Algorithm, minimum sample size, and empirical bias and dispersion , 2013, J. Comput. Appl. Math..

[55]  R. Serfling,et al.  On Invariant Coordinate System (ICS) Functionals , 2012 .