A Bayesian method for identifying independent sources of non-random spatial patterns

A Bayesian blind source separation (BSS) algorithm is proposed in this paper to recover independent sources from observed multivariate spatial patterns. As a widely used mechanism, Gaussian mixture model is adopted to represent the sources for statistical description and machine learning. In the context of linear latent variable BSS model, some conjugate priors are incorporated into the hyperparameters estimation of mixing matrix. The proposed algorithm then approximates the full posteriors over model structure and source parameters in an analytical manner based on variational Bayesian treatment. Experimental studies demonstrate that this Bayesian source separation algorithm is appropriate for systematic spatial pattern analysis by modeling arbitrary sources and identify their effects on high dimensional measurement data. The identified patterns will serve as diagnosis aids for gaining insight into the nature of physical process for the potential use of statistical quality control.

[1]  Lily Lee,et al.  Monitoring Activities from Multiple Video Streams: Establishing a Common Coordinate Frame , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[3]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[5]  Barak A. Pearlmutter,et al.  Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA , 1996, NIPS.

[6]  D. Bartholomew.  Latent Variable Models And Factor Analysis , 1987 .

[7]  J. E. Jackson.  Principal Components and Factor Analysis: Part II - Additional Topics Related to Principal Components , 1981 .

[8]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[9]  Terrence J. Sejnowski,et al.  ICA Mixture Models for Unsupervised Classification of Non-Gaussian Classes and Automatic Context Switching in Blind Signal Separation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Daniel W. Apley,et al.  A Factor-Analysis Method for Diagnosing Variability in Mulitvariate Manufacturing Processes , 2001, Technometrics.

[11]  R. Liu,et al.  AMUSE: a new blind identification algorithm , 1990, IEEE International Symposium on Circuits and Systems.

[12]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[13]  D. Rowe.  A Bayesian approach to blind source separation , 2002 .

[14]  M. Wax,et al.  A least-squares approach to joint diagonalization , 1997, IEEE Signal Processing Letters.

[15]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[16]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[17]  Zoubin Ghahramani,et al.  A Unifying Review of Linear Gaussian Models , 1999, Neural Computation.

[18]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[19]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[20]  Adrian Corduneanu,et al.  Variational Bayesian Model Selection for Mixture Distributions , 2001 .

[21]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Darek Ceglarek,et al.  Fixture Failure Diagnosis for Autobody Assembly Using Pattern Recognition , 1996 .

[23]  Kevin H. Knuth,et al.  Bayesian source separation and localization , 1998, Optics & Photonics.

[24]  Jean-Franois Cardoso.  High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[25]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[26]  I. Jolliffe.  Principal Component Analysis and Factor Analysis , 1986 .

[27]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[28]  Aapo Hyvärinen,et al.  Survey on Independent Component Analysis , 1999 .

[29]  Daniel W. Apley,et al.  Identifying Spatial Variation Patterns in Multivariate Manufacturing Processes , 2003, Technometrics.

[30]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[31]  J. Edward Jackson,et al.  Principal Components and Factor Analysis: Part I - Principal Components , 1980 .