Independent component analysis and beyond

Independent component analysis (ICA) is a tool for statistical data analysis and signal processing that is able to decompose multivariate signals into their underlying source components. Although the classical ICA model is highly useful, there are many real-world applications that require powerful extensions of ICA. This thesis presents new methods that extend the functionality of ICA. Reliability and grouping of independent components with noise injection. Usually noise is considered to be destructive. We present a new method that constructively injects noise to assess the reliability and the grouping structure of empirical independent component estimates. We generalize hereby earlier work on reliability assessment based on bootstrap to arbitrary ICA algorithms. Our method can be viewed as a Monte-Carlo-style approximation of the curvature of some performance measure at the solution. Simulations using artificial and real-world data validate our approach. Robust and overcomplete ICA with inlier detection. Classical ICA algorithms are often sensitive to outliers. We present a new ICA algorithm for super-Gaussian sources that is based on an index for outlier detection that uses nearest neighbor methods. The outlier index is not employed to remove outliers but instead directly to find inliers— the data points in the most concentrated regions—which represent the ICA directions for super-Gaussian source signals. Our inlier-based approach is by construction robust against outliers and can be naturally applied to the overcomplete ICA problem, in which there are more sources than sensors. A comparison of our new method with classical algorithms—in terms of robustness—and a comprehensive empirical analysis of its performance—with respect to dimensionality, number of sources and number of data points—underlines its key advantages. Nonlinear ICA with kernel methods. We present a kernel-based algorithm for non-linear ICA that uses kernel feature spaces to approximate nonlinearities. Applying linear ICA based on time structure in the resulting high-dimensional spaces can un-mix strongly nonlinear mixtures. The key is to use some dimensionality reduction technique to make the application of ICA methods computationally and numerically tractable. Experiments demonstrate the excellent performance and efficiency of our algorithm for several problems of nonlinear ICA. The work of this dissertation has been done at the Fraunhofer institute FIRST (former GMD FIRST) in Berlin. I would like to thank all current and former members of this group for creating an open research atmosphere and for many productive discussions, including Dr. In particular , I thank my roommate Dr. Pavel Laskov who helped me often in many …