Fourth Moments and Independent Component Analysis

In independent component analysis it is assumed that the components of the observed random vector are linear combinations of latent independent random variables, and the aim is then to find an estimate for a transformation matrix back to these independent components. In the engineering literature, there are several traditional estimation procedures based on the use of fourth moments, such as FOBI (fourth order blind identification), JADE (joint approximate diagonalization of eigenmatrices), and FastICA, but the statistical properties of these estimates are not well known. In this paper various independent component functionals based on the fourth moments are discussed in detail, starting with the corresponding optimization problems, deriving the estimating equations and estimation algorithms, and finding asymptotic statistical properties of the estimates. Comparisons of the asymptotic variances of the estimates in wide independent component models show that in most cases JADE and the symmetric version of FastICA perform better than their competitors.

[1]  K. Pearson.  Contributions to the Mathematical Theory of Evolution. II. Skew Variation in Homogeneous Material , 1895 .

[2]  Karl Pearson.  “DAS FEHLERGESETZ UND SEINE VERALLGEMEINER-UNGEN DURCH FECHNER UND PEARSON.” A REJOINDER , 1905 .

[3]  V. Zwet.  Convex transformations of random variables , 1965 .

[4]  Richard B. Darlington,et al.  Is Kurtosis Really “Peakedness?” , 1970 .

[5]  K. Mardia.  Measures of multivariate skewness and kurtosis with applications , 1970 .

[6]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[7]  R. Maronna.  Robust $M$-Estimators of Multivariate Location and Scatter , 1976 .

[8]  R. Serfling.  Approximation Theorems of Mathematical Statistics , 1980 .

[9]  磯貝 恭史.  Measures of multivariate skewness and kurtosis and tests for multivariate normality , 1985 .

[10]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[11]  D. B. Clarkson.  A Least Squares Version of Algorithm as 211: The F‐G Diagonalization Algorithm , 1988 .

[12]  Jean-Francois Cardoso,et al.  Source separation using higher order moments , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[13]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[14]  Tamás F. Móri,et al.  On Multivariate Skewness and Kurtosis , 1994 .

[15]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[16]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[17]  L. T. DeCarlo.  On the meaning and use of kurtosis. , 1997 .

[18]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[19]  F. Prieto,et al.  Cluster Identification Using Projections , 2001 .

[20]  Visa Koivunen,et al.  Blind separation methods based on Pearson system and its extensions , 2002, Signal Process..

[21]  Visa Koivunen,et al.  Identifiability, separability, and uniqueness of linear ICA models , 2004, IEEE Signal Processing Letters.

[22]  E. Oja,et al.  Independent Component Analysis , 2001 .

[23]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[24]  Muni S. Srivastava,et al.  Estimation and Testing of Parameters in Multivariate Laplace Distribution , 2005 .

[25]  P. Tichavský,et al.  Efficient variant of algorithm fastica for independent component analysis attaining the cramer-RAO lower bound , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[26]  Steven J. Schwager,et al.  Multivariate Skewness and Kurtosis , 2006 .

[27]  Erkki Oja,et al.  Efficient Variant of Algorithm FastICA for Independent Component Analysis Attaining the CramÉr-Rao Lower Bound , 2006, IEEE Transactions on Neural Networks.

[28]  Mia Hubert,et al.  Robust measures of tail weight , 2006, Comput. Stat. Data Anal..

[29]  P. Bickel,et al.  Efficient independent component analysis , 2006, 0705.4230.

[30]  Conceição Amado,et al.  Principal Axis Analysis , 2006 .

[31]  Hannu Oja,et al.  Tests of multinormality based on location vectors and scatter matrices , 2007, Stat. Methods Appl..

[32]  S. Bonhomme,et al.  Consistent noisy independent component analysis , 2008 .

[33]  T. Kollo.  Multivariate skewness and kurtosis measures with an application in ICA , 2008 .

[34]  Semiparametrically efficient inference based on signs and ranks for median‐restricted models , 2008 .

[35]  Hannu Oja,et al.  Invariant Coordinate Selection , 2008 .

[36]  David E. Tyler,et al.  Invariant co‐ordinate selection , 2009 .

[37]  John T. Kent,et al.  Independent Component Analysis: An Approach to Clustering , 2009, MSV.

[38]  Pauliina Ilmonen,et al.  Characteristics of multivariate distributions and the invariant coordinate system , 2010 .

[39]  Esa Ollila,et al.  The Deflation-Based FastICA Estimator: Statistical Analysis Revisited , 2010, IEEE Transactions on Signal Processing.

[40]  Klaus Nordhausen,et al.  A New Performance Index for ICA: Properties, Computation and Asymptotic Analysis , 2010, LVA/ICA.

[41]  Francisco J. Prieto,et al.  Eigenvectors of a kurtosis matrix as interesting directions to reveal cluster structure , 2010, J. Multivar. Anal..

[42]  E. Oja,et al.  Performance Analysis of the FastICA Algorithm and Cramér – Rao Bounds for Linear Independent Component Analysis , 2010 .

[43]  Zhang Yan.  Performance Analysis of the FastICA Algorithm in Cochannel Communication System , 2010 .

[44]  Klaus Nordhausen,et al.  Deflation-based FastICA reloaded , 2011, 2011 19th European Signal Processing Conference.

[45]  K. Nordhausen,et al.  Multivariate models and the first four moments , 2011 .

[46]  Pauliina Ilmonen,et al.  Semiparametrically efficient inference based on signed ranks in symmetric independent component models , 2011, 1202.5159.

[47]  M. Yuan,et al.  Independent component analysis via nonparametric maximum likelihood estimation , 2012, 1206.0457.

[48]  Klaus Nordhausen,et al.  Fast equivariant JADE , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[49]  M. Hallin,et al.  R-Estimation for Asymmetric Independent Component Analysis , 2013, 1304.3073.

[50]  Klaus Nordhausen,et al.  Deflation-Based FastICA With Adaptive Choices of Nonlinearities , 2014, IEEE Transactions on Signal Processing.

[51]  Klaus Nordhausen,et al.  Separation of Uncorrelated Stationary time series using Autocovariance Matrices , 2014, Journal of Time Series Analysis.

[52]  Hannu Oja,et al.  On Location, Scale, Skewness and Kurtosis of Univariate Distributions , 2016 .

[53]  H. Oja,et al.  Scatter Matrices and Independent Component Analysis , 2006 .