Nonlinear Independent Factor Analysis by Hierarchical Models

The building blocks introduced earlier by us in [1] are used for constructing a hierarchical nonlinear model for nonlinear factor analysis. We call the resulting method hierarchical nonlinear factor analysis (HNFA). The variational Bayesian learning algorithm used in this method has a linear computational complexity, and it is able to infer the structure of the model in addition to estimating the unknown parameters. We show how nonlinear mixtures can be separated by first estimating a nonlinear subspace using HNFA and then rotating the subspace using linear independent component analysis. Experimental results show that the cost function minimised during learning predicts well the quality of the estimated subspace.

[1]  Charles M. Bishop,et al.  Ensemble learning in Bayesian neural networks , 1998 .

[2]  Aapo Hyvärinen,et al.  Nonlinear independent component analysis: Existence and uniqueness results , 1999, Neural Networks.

[3]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[4]  Harri Lappalainen,et al.  Ensemble learning for independent component analysis , 1999 .

[5]  Stephen J. Roberts,et al.  An ensemble learning approach to independent component analysis , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[6]  Antti Honkela,et al.  Bayesian Non-Linear Independent Component Analysis by Multi-Layer Perceptrons , 2000 .

[7]  Nikunj C. Oza,et al.  Online Ensemble Learning , 2000, AAAI/IAAI.

[8]  David J. C. MacKay,et al.  Ensemble Learning for Blind Image Separation and Deconvolution , 2000 .

[9]  H. Attias.  Independent Component Analysis: ICA, graphical models and variational methods , 2001 .

[10]  Matthew J. Beal,et al.  Graphical Models and Variational Methods , 2001 .

[11]  E. Oja,et al.  Independent Component Analysis , 2013 .

[12]  J. Karhunen,et al.  Building Blocks for Hierarchical Latent Variable Models , 2001 .

[13]  Terrence J. Sejnowski,et al.  Variational Learning of Clusters of Undercomplete Nonsymmetric Independent Components , 2003, J. Mach. Learn. Res..

[14]  A. Honkela.  Speeding up cyclic update schemes by pattern searches , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[15]  Juha Karhunen,et al.  An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models , 2002, Neural Computation.

[16]  J. Karhunen,et al.  Advances in Nonlinear Blind Source Separation , 2003 .

[17]  Alexander Ilin,et al.  On the Effect of the Form of the Posterior Approximation in Variational Learning of ICA Models , 2005, Neural Processing Letters.