Hybridizing Genetic Algorithms with ICA in Higher Dimension

In this paper we present a novel method for blindly separating unobservable independent component signals from their linear mixtures, using genetic algorithms (GA) to minimize the nonconvex and nonlinear cost functions. This approach is very useful in many fields such as forecasting indexes in financial stock markets where the search for independent components is the major task to include exogenous information into the learning machine. The GA presented in this work is able to extract independent components with faster rate than the previous independent component analysis algorithms based on Higher Order Statistics (HOS) as input space dimension increases showing significant accuracy and robustness.

[1]  C.G. Puntonet,et al.  3D spatial analysis of fMRI data: a comparison of ICA and GLM analysis on a word perception task , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[2]  Anke Meyer-Bäse,et al.  Application of unsupervised clustering methods to biomedical image analysis , 2005 .

[3]  Fabian J. Theis,et al.  SOMICA - an application of self-organizing maps to geometric independent component analysis , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[4]  Fabian J. Theis,et al.  A Theoretical Framework for Overcomplete Geometric BMMR , 2002 .

[5]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[6]  Helge J. Ritter,et al.  Clustering of Dependent Components: A New Paradigm for fMRI Signal Detection , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[7]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[8]  A. E. Eiben,et al.  Global Convergence of Genetic Algorithms: A Markov Chain Analysis , 1990, PPSN.

[9]  Fabian J. Theis,et al.  A Geometric ICA Procedure Based on a Lattice of the Observation Space , 2003 .

[10]  Fabian J. Theis,et al.  Connecting geometric independent component analysis to unsupervised learning algorithms , 2004 .

[11]  T. Saunders,et al.  Theses , 2001 .

[12]  Fabian J. Theis,et al.  Blind Source Separation of Water Artefacts in NMR Spectra using a Matrix Pencil , 2003 .

[13]  Fabian J. Theis,et al.  A Hybridization of Simulated Annealing and Local PCA for Automatic Component Assignment Within ICA , 2005, IWANN.

[14]  Fabian J Theis,et al.  Local features in biomedical image clusters extracted with independent component analysis , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[15]  Fabian J Theis,et al.  Blind sensor characteristics estimation in a multi-sensor network applied to fMRI analysis , 2004, Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, 2004..

[16]  Fabian J. Theis,et al.  Linear Geometric ICA: Fundamentals and Algorithms , 2003, Neural Computation.

[17]  Carlos García Puntonet,et al.  Applying Neural Networks and Genetic Algorithms to the Separation of Sources , 2002, IBERAMIA.

[18]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[19]  Fabian J. Theis,et al.  Sparse Representation of Data and Support Vector Machines (in: Proceedings) , 2004 .

[20]  Fabian J. Theis,et al.  Sparse Component Analysis: a New Tool for Data Mining , 2004 .

[21]  ' F.Rojas,et al.  A NEW ICA METHOD BASED ON A LATTICE OF THE OBSERVATION SPACE , 2004 .

[22]  Fabian J Theis,et al.  Kernel-PCA denoising of artifact-free protein NMR spectra , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[23]  Fabian J Theis,et al.  Adaptive signal analysis of immunological data , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[24]  A.M. Tome,et al.  An algorithm for automatic assignment of artifact-related independent components in biomedical signal analysis , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[25]  C. L. Nikias,et al.  Higher-order spectra analysis : a nonlinear signal processing framework , 1993 .

[26]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[27]  Fabian J. Theis.  Geometric ICA in overcomplete and high-dimensional settings , 2002 .

[28]  Fabian J. Theis,et al.  SOMICA and geometric ICA , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[29]  Fabian J. Theis,et al.  Separability of analytic postnonlinear blind source separation with bounded sources , 2004, ESANN.

[30]  Fabian J. Theis,et al.  New Geometric ICA Approach for Blind Source Separation , 2003 .

[31]  Fabian J. Theis,et al.  AutoAssign - An Automatic Assignment Tool for Independent Components , 2005, IbPRIA.

[32]  Elmar Lang,et al.  A Matrix Pencil Approach to the Blind Source Separation of Artifacts in 2D NMR Spectra , 2003 .

[33]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .

[34]  Fabian J. Theis,et al.  How to generalize geometric ICA to higher dimensions , 2002, ESANN.

[35]  Fabian J. Theis,et al.  Linearization identification and an application to BSS using a SOM , 2004, ESANN.

[36]  Juan Manuel Górriz Sáez.  Algoritmos híbridos para la modelización de series temporales con técnicas AR-ICA , 2003 .

[37]  Olle Häggström.  Finite Markov Chains and Algorithmic Applications , 2002 .

[38]  Fabian J. Theis,et al.  Quadratic Independent Component Analysis , 2004 .

[39]  Fabian J. Theis,et al.  Mathematics in independent component analysis , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[40]  Edward L. Shaughnessy.  Abstracts , 2001, Veterinary Record.

[41]  Ruey-Wen Liu,et al.  General approach to blind source separation , 1996, IEEE Trans. Signal Process..

[42]  Miguel Toro,et al.  Advances in Artificial Intelligence — IBERAMIA 2002 , 2002, Lecture Notes in Computer Science.

[43]  Fabian J. Theis,et al.  Extending Geometric ICA to Overcomplete and High-Dimensional BSS-Problems , 2002 .

[44]  Fabian J. Theis,et al.  Functional MRI Analysis by a Novel Spatiotemporal ICA Algorithm , 2005, ICANN.

[45]  Fabian J. Theis,et al.  Comparison of maximum entropy and minimal mutual information in a nonlinear setting , 2002, Signal Process..

[46]  Fabian J. Theis,et al.  Extended Sparse Nonnegative Matrix Factorization , 2005, IWANN.

[47]  Fabian J. Theis,et al.  3D Spatial Analysis of fMRI Data on a Word Perception Task , 2004, ICA.

[48]  Toshihisa Tanaka,et al.  A Fast and Efficient Method for Compressing fMRI Data Sets , 2005, ICANN.

[49]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[50]  Fabian J Theis,et al.  Uniqueness of linear independent component analysis and the Skitovitch-Darmois theorem , 2003 .

[51]  Fabian J. Theis,et al.  Spatial ICA of fMRI data in time windows , 2004 .

[52]  Anke Meyer-Bäse,et al.  Spatiotemporal blind source separation using double-sided approximate joint diagonalization , 2005, 2005 13th European Signal Processing Conference.

[53]  Fabian J. Theis,et al.  Neural network signal analysis in immunology , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[54]  Fabian J. Theis,et al.  Blind source recovery: algorithm comparison and fusion , 2004 .

[55]  Pedro Larrañaga,et al.  Genetic Algorithms: Bridging the Convergence Gap , 1999, Theor. Comput. Sci..

[56]  Fabian J. Theis,et al.  A HISTOGRAM-BASED OVERCOMPLETE ICA ALGORITHM , 2003 .

[57]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[58]  Fabian J. Theis,et al.  Removing water artefacts from 2D protein NMR spectra using GEVD with congruent matrix pencils , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[59]  Fabian J Theis,et al.  On the use of independent component analysis to remove water artefacts of 2D NMR protein spectra , 2003 .

[60]  Fabian J. Theis,et al.  Geometric overcomplete ICA , 2002, ESANN.

[61]  Joe Suzuki,et al.  A Markov chain analysis on simple genetic algorithms , 1995, IEEE Trans. Syst. Man Cybern..

[62]  Chrystopher L. Nehaniv,et al.  Linear Analysis of Genetic Algorithms , 1998, Theor. Comput. Sci..

[63]  Carlos G. Puntonet,et al.  Neural net approach for blind separation of sources based on geometric properties , 1998, Neurocomputing.

[64]  Fabian J. Theis,et al.  Robust overcomplete matrix recovery for sparse sources using a generalized Hough transform , 2004, ESANN.

[65]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[66]  Fabian J. Theis,et al.  Uniqueness of real and complex linear independent component analysis revisited , 2004, 2004 12th European Signal Processing Conference.

[67]  Fabian J. Theis,et al.  AUTOMATIC DENOISING USING LOCAL INDEPENDENT COMPONENT ANALYSIS , 2004 .

[68]  Fabian J. Theis,et al.  Maximum Entropy and Minimal Mutual Information in a Nonlinear Model , 2001 .

[69]  Fabian J. Theis,et al.  Information-theoretic biomedical data analysis , 2004 .

[70]  Fabian J. Theis,et al.  Optimization Algorithms for Sparse Representations and Applications , 2006 .

[71]  Fabian J. Theis,et al.  A new geometrical method of BSS on a lattice of the space of observations , 2003 .

[72]  Fabian J. Theis,et al.  Nonlinear Geometric ICA , 2003 .

[73]  Fabian J. Theis,et al.  Automated counting of labelled cells in rodent brain section images , 2004 .

[74]  Jun Wang,et al.  Nonlinear Blind Source Separation Using Higher Order Statistics and a Genetic Algorithm , 2001 .

[75]  Andrew D. Back,et al.  A First Application of Independent Component Analysis to Extracting Structure from Stock Returns , 1997, Int. J. Neural Syst..

[76]  Fabian J. Theis,et al.  ZANE - an algorithm for counting labelled cells in section images , 2004 .

[77]  Fabian J. Theis,et al.  Blind Source Separation of Linear Mixtures with Singular Matrices , 2004, ICA.

[78]  Fabian J. Theis,et al.  Bispectrum-Based Statistical Tests for VAD , 2005, ICANN.

[79]  Fabian J Theis,et al.  Denoising using local ICA and kernel-PCA , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[80]  Fabian J. Theis,et al.  FastGeo - A Histogram Based Approach to Linear Geometric ICA , 2001 .

[81]  Fabian J. Theis,et al.  Geometric source separation: algorithms and applications , 2003 .

[82]  Fabian J Theis,et al.  Postnonlinear blind source separation via linearization identification , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[83]  Closing Boundary Components and Seifert Manifolds in the o-Graph Calculus , 2002 .

[84]  M. Lautenschlager.  Topological Constructions in the o-Graph Calculus , 2000 .

[85]  Fabian J Theis,et al.  Formalization of the Two-Step Approach to Overcomplete BSS , 2002 .

[86]  Fabian J. Theis,et al.  A geometric algorithm for overcomplete linear ICA , 2004, Neurocomputing.