Alternative Derivation of FastICA With Novel Power Iteration Algorithm

The widely used fixed-point FastICA algorithm has been derived and motivated as being an approximate Newton-Raphson (NR) algorithm. In the original derivation, the Lagrangian multiplier is treated as a constant and an ad hoc approximation is used for Jacobian matrix in the NR update. In this letter, we provide an alternative derivation of the FastICA algorithm without approximation. We show that any solution to the FastICA algorithm is a solution to the exact NR algorithm as well. In addition, we propose a novel power iteration algorithm for FastICA which is remarkably more stable than the fixed-point algorithm, when the sample size is not orders of magnitudes larger than the dimension. Our proposed algorithm can be run on parallel computing nodes.

[1]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[2]  E. Oja,et al.  Performance Analysis of the FastICA Algorithm and Cramér – Rao Bounds for Linear Independent Component Analysis , 2010 .

[3]  Klaus Nordhausen,et al.  Deflation-Based FastICA With Adaptive Choices of Nonlinearities , 2014, IEEE Transactions on Signal Processing.

[4]  Esa Ollila,et al.  The Deflation-Based FastICA Estimator: Statistical Analysis Revisited , 2010, IEEE Transactions on Signal Processing.

[5]  J. Sherman,et al.  Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given Matrix , 1950 .

[6]  Richard A. Tapia,et al.  Inverse, Shifted Inverse, and Rayleigh Quotient Iteration as Newton's Method , 2018, SIAM Rev..

[7]  Visa Koivunen,et al.  Compact CramÉr–Rao Bound Expression for Independent Component Analysis , 2008, IEEE Transactions on Signal Processing.

[8]  Scott C. Douglas.  Relationships Between the FastICA Algorithm and the Rayleigh Quotient Iteration , 2006, ICA.

[9]  Hao Shen,et al.  On the Relationships Between Power Iteration, Inverse Iteration and FastICA , 2007, ICA.

[10]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[11]  Erkki Oja,et al.  Performance analysis of the FastICA algorithm and Crame/spl acute/r-rao bounds for linear independent component analysis , 2006, IEEE Transactions on Signal Processing.

[12]  H. Benaroya,et al.  Probability Models in Engineering and Science , 2005 .

[13]  G. Golub,et al.  Eigenvalue computation in the 20th century , 2000 .