Mixtures of multivariate restricted skew-normal factor analyzer models in a Bayesian framework

AbstractThe mixture of factor analyzers (MFA) model, by reducing the number of free parameters through its factor-analytic representation of the component covariance matrices, is an important statistical model to identify hidden or latent groups in high dimensional data. Recent approaches to extend the approach to skewed data or skewness in the latent groups have been examined in a frequentist setting where there are some known computational limitations. For these reasons we consider a Bayesian approach to the restricted skew-normal mixtures of factor analysis MFA model. We examine the performance and flexibility of the approach on real datasets and illustrate some of the computational advantages in a missing data setting.

[1]  L. Carin,et al.  Predicting Viral Infection From High-Dimensional Biomarker Trajectories , 2011, Journal of the American Statistical Association.

[2]  Kerrie Mengersen,et al.  Mixtures: Estimation and Applications , 2011 .

[3]  A Robust Bayesian Approach for Structural Equation Models with Missing Data , 2008 .

[4]  A. Azzalini,et al.  The multivariate skew-normal distribution , 1996 .

[5]  D. Dunson,et al.  Bayesian Semiparametric Structural Equation Models with Latent Variables , 2010 .

[6]  Xiao-Li Meng,et al.  The Art of Data Augmentation , 2001 .

[7]  Zoubin Ghahramani,et al.  Variational Inference for Bayesian Mixtures of Factor Analysers , 1999, NIPS.

[8]  Zoubin Ghahramani,et al.  Infinite Sparse Factor Analysis and Infinite Independent Components Analysis , 2007, ICA.

[9]  David B Dunson,et al.  Default Prior Distributions and Efficient Posterior Computation in Bayesian Factor Analysis , 2009, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[10]  Piotr A. Kowalski,et al.  Complete Gradient Clustering Algorithm for Features Analysis of X-Ray Images , 2010 .

[11]  Xin-Yuan Song,et al.  Semiparametric Bayesian analysis of structural equation models with fixed covariates , 2008, Statistics in medicine.

[12]  Reinaldo Boris Arellano-Valle,et al.  Robust finite mixture modeling of multivariate unrestricted skew-normal generalized hyperbolic distributions , 2019, Stat. Comput..

[13]  C. Robert,et al.  Deviance information criteria for missing data models , 2006 .

[14]  M. Drton,et al.  Order-invariant prior specification in Bayesian factor analysis , 2014, 1409.7672.

[15]  D. Rubin,et al.  Statistical Analysis with Missing Data. , 1989 .

[16]  Yaming Yu,et al.  To Center or Not to Center: That Is Not the Question—An Ancillarity–Sufficiency Interweaving Strategy (ASIS) for Boosting MCMC Efficiency , 2011 .

[17]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[18]  Michael A. West,et al.  BAYESIAN MODEL ASSESSMENT IN FACTOR ANALYSIS , 2004 .

[19]  Christopher M. Bishop,et al.  Bayesian PCA , 1998, NIPS.

[20]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[21]  D. M. Titterington,et al.  Mixtures of Factor Analysers. Bayesian Estimation and Inference by Stochastic Simulation , 2004, Machine Learning.

[22]  Thomas J. Leininger,et al.  Bayesian Inference and Model Assessment for Spatial Point Patterns Using Posterior Predictive Samples , 2017 .

[23]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[24]  Victor H. Lachos,et al.  Robust mixture modeling based on scale mixtures of skew-normal distributions , 2010, Comput. Stat. Data Anal..

[25]  A. Azzalini.  A class of distributions which includes the normal ones , 1985 .

[26]  Adelchi Azzalini,et al.  The Skew-Normal and Related Families , 2018 .

[27]  David B. Dunson,et al.  Compressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds , 2010, IEEE Transactions on Signal Processing.

[28]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[29]  R. Arellano-Valle,et al.  Maximum a-posteriori estimation of autoregressive processes based on finite mixtures of scale-mixtures of skew-normal distributions , 2017 .

[30]  Lawrence Carin,et al.  Nonparametric factor analysis with beta process priors , 2009, ICML '09.

[31]  Hedibert Freitas Lopes,et al.  Parsimonious Bayesian Factor Analysis when the Number of Factors is Unknown , 2010 .

[32]  Ryan P. Browne,et al.  Mixtures of skew-t factor analyzers , 2013, Comput. Stat. Data Anal..

[33]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[34]  Reinaldo B. Arellano-Valle,et al.  A flexible class of parametric distributions for Bayesian linear mixed models , 2018, TEST.

[35]  Tomohiro Ando,et al.  Bayesian factor analysis with fat-tailed factors and its exact marginal likelihood , 2009, J. Multivar. Anal..

[36]  M. West,et al.  High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics , 2008, Journal of the American Statistical Association.

[37]  R. Arellano-Valle,et al.  On the Unification of Families of Skew‐normal Distributions , 2006 .

[38]  Subhashis Ghosal,et al.  Bayesian Estimation of Principal Components for Functional Data , 2017 .

[39]  Tsung-I Lin,et al.  Finite mixture modelling using the skew normal distribution , 2007 .

[40]  MARGINAL MARKOV CHAIN MONTE CARLO METHODS , 2010 .

[41]  Yasuo Amemiya,et al.  Mixture Factor Analysis for Approximating a Nonnormally Distributed Continuous Latent Factor With Continuous and Dichotomous Observed Variables , 2012, Multivariate behavioral research.

[42]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[43]  Xiao-Li Meng,et al.  Seeking efficient data augmentation schemes via conditional and marginal augmentation , 1999 .

[44]  D. Dunson,et al.  Sparse Bayesian infinite factor models. , 2011, Biometrika.

[45]  Geoffrey J. McLachlan,et al.  Model-based clustering and classification with non-normal mixture distributions , 2013, Stat. Methods Appl..

[46]  S. Sahu,et al.  A new class of multivariate skew distributions with applications to bayesian regression models , 2003 .

[47]  Jared S. Murray,et al.  Bayesian Gaussian Copula Factor Models for Mixed Data , 2011, Journal of the American Statistical Association.

[48]  Javier E. Contreras-Reyes,et al.  The Skew-Reflected-Gompertz distribution for analyzing symmetric and asymmetric data , 2019, J. Comput. Appl. Math..

[49]  Geoffrey J. McLachlan,et al.  On mixtures of skew normal and skew $$t$$-distributions , 2012, Adv. Data Anal. Classif..

[50]  J. Heckman,et al.  Bayesian Exploratory Factor Analysis , 2014, Journal of econometrics.

[51]  C. Robert,et al.  Computational and Inferential Difficulties with Mixture Posterior Distributions , 2000 .

[52]  A. Utsugi,et al.  Bayesian Analysis of Mixtures of Factor Analyzers , 2001, Neural Computation.

[53]  Geoffrey J. McLachlan,et al.  Extending mixtures of factor models using the restricted multivariate skew-normal distribution , 2013, J. Multivar. Anal..

[54]  M. Maleki,et al.  Two-Piece location-scale distributions based on scale mixtures of normal family , 2017 .

[55]  A. Azzalini,et al.  Statistical applications of the multivariate skew normal distribution , 2009, 0911.2093.

[56]  Bradley P. Carlin,et al.  Bayesian Methods for Data Analysis , 2008 .

[57]  Xin-Yuan Song,et al.  A semiparametric Bayesian approach for structural equation models , 2010, Biometrical journal. Biometrische Zeitschrift.