Neurally plausible, non-combinatorial iterative independent process analysis

It has been shown recently that the identification of mixed hidden independent auto-regressive processes (independent process analysis, IPA), under certain conditions, can be free from combinatorial explosion. The key is that IPA can be reduced (i) to independent subspace analysis and then, via a novel decomposition technique called Separation Theorem, (ii) to independent component analysis. Here, we introduce an iterative scheme and its neural network representation that takes advantage of the reduction method and can accomplish the IPA task. Computer simulation illustrates the working of the algorithm.

[1]  Hagai Attias,et al.  An EM Method for Spatio-temporal Blind Source Separation Using an AR-MOG Source Model , 2006, ICA.

[2]  Fabian J. Theis,et al.  Blind signal separation into groups of dependent signals using joint block diagonalization , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[3]  P. Kumar,et al.  Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[4]  Barnabás Póczos,et al.  Separation Theorem for Independent Subspace Analysis , 2005 .

[5]  Aapo Hyvärinen,et al.  Independent Component Analysis for Time-dependent Stochastic Processes , 1998 .

[6]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[7]  Aapo Hyvärinen,et al.  Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation , 1999, Neural Computation.

[8]  E. Oja,et al.  Independent Component Analysis , 2013 .

[9]  Barnabás Póczos,et al.  Cross-Entropy Optimization for Independent Process Analysis , 2006, ICA.

[10]  Lei Xu,et al.  Dual multivariate auto-regressive modeling in state space for temporal signal separation , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[11]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[12]  Barnabás Póczos,et al.  Non-combinatorial estimation of independent autoregressive sources , 2006, Neurocomputing.

[13]  Barnabas Poczos,et al.  Theorem for K-Independent Subspace Analysis with Sufficient Conditions , 2006, math/0608100.

[14]  Barnabás Póczos,et al.  Independent Subspace Analysis on Innovations , 2005, ECML.