Characterization of different datasets for ICA algorithms

There are several Independent Component Analysis (ICA) algorithms based on different approaches that are used to estimate the independent components, up to some precision, from a linear mixture of independent components as long as the independent components do not follow Gaussian distribution. Some approaches, however, work better than the other if data distribution and characteristics follow a certain pattern. From the mixture of data comprising two or more independent components it is quite hard, if not impossible, to find out the distribution of the independent components accurately and therefore to characterize one or more ICA approaches to be better than others for certain type of data. This paper describes a framework for ICA algorithms as proposed by Ejaz [1]. In this study we have characterized four different ICA algorithms, with some pre-selected fixed parameters, based on different approaches for a number of different datasets that are linearly mixed with two to five independent components that follow a number of different distributions. ICA algorithms used for the research include FastICA, Extended Infomax, JADE, and Kernel ICA based on canonical correlation analysis. All of these algorithms are discussed briefly yet covering most of their important aspects such that this paper also serves as a tutorial for these ICA algorithms for novice readers. We have done an extensive statistical analysis for more than 300 different datasets to characterize them for one or more of the four algorithms based on to which algorithm estimates the independent components closest to the original components.

[1]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[2]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[3]  P.L.Lai,et al.  A neural network implementation of canonical correlation analysis , 1999 .

[4]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[5]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[6]  J. Wishart,et al.  Statistics in Research. , 1956 .

[7]  D. M. Etter,et al.  Engineering Problem Solving with Matlab , 1993 .

[8]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[9]  S. Leigh,et al.  Probability and Random Processes for Electrical Engineering , 1989 .

[10]  Pei Ling Lai,et al.  Ica Using Kernel Canonical Correlation Analysis , 2000 .

[11]  Aapo Hyvärinen,et al.  Bubbles: a unifying framework for low-level statistical properties of natural image sequences. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[12]  Fabian J. Theis,et al.  Mathematics in independent component analysis , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[13]  Simon Haykin,et al.  Neural networks , 1994 .

[14]  H. Rutishauser The Jacobi method for real symmetric matrices , 1966 .

[15]  K. Clint Slatton,et al.  A parameterized statistical sonar image texture model , 2008, SPIE Defense + Commercial Sensing.

[17]  Aapo Hyvärinen,et al.  Statistical models of images and early vision , 2005 .

[18]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[19]  A. Floren,et al.  ' " ' " ' " . " ' " " " " " ' " ' " " " " " : ' " 1 , 2001 .

[20]  J. Besag Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[21]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[22]  Colin Fyfe,et al.  A neural implementation of canonical correlation analysis , 1999, Neural Networks.