Independent component analysis employing exponentials of sparse antisymmetric matrices

Abstract Independent component analysis (ICA) is a standard method for separating a multivariate signal into additive components that are non-Gaussian and independent from each other. This paper introduced a novel algorithm to perform ICA employing matrix exponentials, which performs similarly to geodesic based methods but based on a different insight. First, we showed that the ICA problem can be formulated as an optimization problem in the space of orthogonal matrices whose determinants are one, which can be further transformed into an equivalent problem in the space of antisymmetric matrices. Then, an efficient approach was presented for iteratively solving this problem using the antisymmetric matrices with one or more nonzero columns and rows. Especially, we proved that in the sense of local optimization it is sufficient to employ antisymmetric matrices with only one nonzero column and row. The analytical expressions of exponentials of such special antisymmetric matrices were also explicitly established in this paper. Compared to other competing algorithms, experimental results indicated that the proposed method can achieve separation with superior performance in term of the precision and running speed.

[1]  Duncan Fyfe Gillies,et al.  A Review of Feature Selection and Feature Extraction Methods Applied on Microarray Data , 2015, Adv. Bioinformatics.

[2]  Bram B. Zandbelt,et al.  Common and unique neural networks for proactive and reactive response inhibition revealed by independent component analysis of functional MRI data , 2014, NeuroImage.

[3]  Tzyy-Ping Jung,et al.  Real-Time Adaptive EEG Source Separation Using Online Recursive Independent Component Analysis , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Cleve B. Moler,et al.  Nineteen Dubious Ways to Compute the Exponential of a Matrix, Twenty-Five Years Later , 1978, SIAM Rev..

[5]  Pierre Comon,et al.  Independent component analysis, a survey of some algebraic methods , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[6]  Swati Goel,et al.  ICA in Image Processing: A Survey , 2015, 2015 IEEE International Conference on Computational Intelligence & Communication Technology.

[7]  Pedram Ghamisi,et al.  Fusion of Hyperspectral and LiDAR Data Using Sparse and Low-Rank Component Analysis , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[8]  J. Gallier,et al.  COMPUTING EXPONENTIALS OF SKEW-SYMMETRIC MATRICES AND LOGARITHMS OF ORTHOGONAL MATRICES , 2002 .

[9]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[10]  Simone G. O. Fiori,et al.  Quasi-Geodesic Neural Learning Algorithms Over the Orthogonal Group: A Tutorial , 2005, J. Mach. Learn. Res..

[11]  N. R. Sakthivel,et al.  Evaluation of expert system for condition monitoring of a single point cutting tool using principle component analysis and decision tree algorithm , 2011, Expert Syst. Appl..

[12]  Jianping Li,et al.  Normalized natural gradient in independent component analysis , 2010, Signal Process..

[13]  Ata Kabán,et al.  Topic Identification in Dynamical Text by Complexity Pursuit , 2003, Neural Processing Letters.

[14]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[15]  Ella Bingham,et al.  ICA and SOM in text document analysis , 2002, SIGIR '02.

[16]  U. Grenander,et al.  Jump–diffusion Markov processes on orthogonal groups for object pose estimation , 2002 .

[17]  Tülay Adalı,et al.  Diversity in Independent Component and Vector Analyses: Identifiability, algorithms, and applications in medical imaging , 2014, IEEE Signal Processing Magazine.

[18]  Shyh-Jye Jou,et al.  Blind ICA detection based on second-order cone programming for MC-CDMA systems , 2014, EURASIP J. Adv. Signal Process..

[19]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[20]  B. Ellingson,et al.  Evaluation of Encephaloduroarteriosynangiosis Efficacy Using Probabilistic Independent Component Analysis Applied to Dynamic Susceptibility Contrast Perfusion MRI , 2017, American Journal of Neuroradiology.

[21]  Anuj Srivastava,et al.  Optimal linear representations of images for object recognition , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  Gang Wang,et al.  The Removal of EOG Artifacts From EEG Signals Using Independent Component Analysis and Multivariate Empirical Mode Decomposition , 2016, IEEE Journal of Biomedical and Health Informatics.

[23]  Steven Thomas Smith,et al.  Optimization Techniques on Riemannian Manifolds , 2014, ArXiv.

[24]  Khaldoun Khashanah,et al.  The Reconstruction of Financial Signals Using Fast ICA for Systemic Risk , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[25]  Shun-ichi Amari,et al.  Unified stabilization approach to principal and minor components extraction algorithms , 2001, Neural Networks.

[26]  Anne Greenbaum,et al.  Using Nonorthogonal Lanczos Vectors in the Computation of Matrix Functions , 1998, SIAM J. Sci. Comput..

[27]  Jacek Tabor,et al.  ICA based on asymmetry , 2017, Pattern Recognit..

[28]  Manoranjan Paul,et al.  Impact of ICA-Based Image Enhancement Technique on Retinal Blood Vessels Segmentation , 2018, IEEE Access.

[29]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[30]  Yasunori Nishimori,et al.  Learning algorithm for independent component analysis by geodesic flows on orthogonal group , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[31]  Jon Atli Benediktsson,et al.  Spectral and Spatial Classification of Hyperspectral Images Based on ICA and Reduced Morphological Attribute Profiles , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Fathi M. Salem,et al.  Adaptive Blind CDMA Receivers Based on ICA Filtered Structures , 2017, Circuits Syst. Signal Process..

[33]  Vicente Zarzoso,et al.  On the Link Between L1-PCA and ICA , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Feng Zhou,et al.  Tensorial Independent Component Analysis-Based Feature Extraction for Polarimetric SAR Data Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Mark A. Girolami,et al.  An Alternative Perspective on Adaptive Independent Component Analysis Algorithms , 1998, Neural Computation.

[36]  P. Absil,et al.  Riemannian Geometry of Grassmann Manifolds with a View on Algorithmic Computation , 2004 .

[37]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[38]  C. Udriste,et al.  Convex Functions and Optimization Methods on Riemannian Manifolds , 1994 .

[39]  Scott C. Douglas,et al.  Self-stabilized gradient algorithms for blind source separation with orthogonality constraints , 2000, IEEE Trans. Neural Networks Learn. Syst..

[40]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[41]  Ying Nian Wu,et al.  Decoding the encoding of functional brain networks: An fMRI classification comparison of non-negative matrix factorization (NMF), independent component analysis (ICA), and sparse coding algorithms , 2016, Journal of Neuroscience Methods.

[42]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[43]  Simone G. O. Fiori,et al.  A Theory for Learning by Weight Flow on Stiefel-Grassman Manifold , 2001, Neural Computation.

[44]  Laurent Albera,et al.  Efficient Stepsize Selection Strategy for Givens Parametrized ICA Applied to EEG Denoising , 2017, IEEE Signal Processing Letters.

[45]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[46]  Simone G. O. Fiori,et al.  A theory for learning based on rigid bodies dynamics , 2002, IEEE Trans. Neural Networks.

[47]  Colin Fyfe,et al.  An extended exploratory projection pursuit network with linear and nonlinear anti-hebbian lateral connections applied to the cocktail party problem , 1997, Neural Networks.

[48]  Andrzej Cichocki,et al.  Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Jun Guo,et al.  Feature selection for neutral vector in EEG signal classification , 2016, Neurocomputing.

[50]  Zied Malouche,et al.  Extended anti-Hebbian adaptation for unsupervised source extraction , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[51]  Ludovica Griffanti,et al.  Automatic denoising of functional MRI data: Combining independent component analysis and hierarchical fusion of classifiers , 2014, NeuroImage.

[52]  David B. Skillicorn,et al.  Automated Concept Location Using Independent Component Analysis , 2008, 2008 15th Working Conference on Reverse Engineering.

[53]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[54]  Mark D. Plumbley.  Geometrical methods for non-negative ICA: Manifolds, Lie groups and toral subalgebras , 2005, Neurocomputing.

[55]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[56]  Mark D. Plumbley.  Algorithms for nonnegative independent component analysis , 2003, IEEE Trans. Neural Networks.

[57]  Visa Koivunen,et al.  Enhanced bootstrap method for statistical inference in the ICA model , 2017, Signal Process..

[58]  S. Shankar Sastry,et al.  Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation , 2001, International Journal of Computer Vision.

[59]  Jonathan H. Manton,et al.  Optimization algorithms exploiting unitary constraints , 2002, IEEE Trans. Signal Process..

[60]  Shotaro Akaho,et al.  Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold , 2005, Neurocomputing.

[61]  Nicoletta Del Buono,et al.  A Survey on Methods for Computing Matrix Exponentials in Numerical Schemes for ODEs , 2003, International Conference on Computational Science.

[62]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..