Bayesian Information Criterion and Selection of the Number of Factors in Factor Analysis Models

In maximum likelihood exploratory factor analysis, the estimates of unique variances can often turn out to be zero or negative, which makes no sense from a statistical point of view. In order to overcome this diculty, we employ a Bayesian approach by specifying a prior distribution for the variances of unique factors. The factor analysis model is estimated by EM algorithm, for which we provide the expectation and maximization steps within a general framework of EM algorithms. Crucial issues in Bayesian factor analysis model are the choice of adjusted parameters including the number of factors and also the hyper-parameters for the prior distribution. The choice of these parameters can be viewed as a model selection and evaluation problem. We derive a model selection criterion for evaluating a Bayesian factor analysis model. Monte Carlo simulations are conducted to investigate the eectiveness of the proposed procedure. A real data example is also given to illustrate our procedure. We observe that our modeling procedure prevents the occurrence of improper solutions and also chooses the appropriate number of factors objectively.

[1]  Herman Rubin,et al.  Statistical Inference in Factor Analysis , 1956 .

[2]  H. Kaiser.  The varimax criterion for analytic rotation in factor analysis , 1958 .

[3]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[4]  A. E. Maxwell,et al.  Factor Analysis as a Statistical Method. , 1964 .

[5]  K. Jöreskog.  Some contributions to maximum likelihood factor analysis , 1967 .

[6]  Stephen M. Robinson,et al.  A Newton-Raphson algorithm for maximum likelihood factor analysis , 1969 .

[7]  M. R. B. Clarke,et al.  A RAPIDLY CONVERGENT METHOD FOR MAXIMUM-LIKELIHOOD FACTOR ANALYSIS , 1970 .

[8]  J. C. Gower,et al.  Factor Analysis as a Statistical Method. 2nd ed. , 1972 .

[9]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[10]  R. P. McDonald,et al.  Bayesian estimation in unrestricted factor analysis: A treatment for heywood cases , 1975 .

[11]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[12]  O. P. V. Driel,et al.  On various causes of improper solutions in maximum likelihood factor analysis , 1978 .

[13]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .

[14]  Pranab Kumar Sen,et al.  Multivariate Analysis (2nd ed.). , 1983 .

[15]  Donald B. Rubin,et al.  More on EM for ML factor analysis , 1983 .

[16]  J. S. Tanaka,et al.  Problems with EM algorithms for ML factor analysis , 1983 .

[17]  James C. Anderson,et al.  The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis , 1984 .

[18]  S. J. Press,et al.  Applied multivariate analysis : using Bayesian and frequentist methods of inference , 1984 .

[19]  A. Boomsma.  Nonconvergence, improper solutions, and starting values in lisrel maximum likelihood estimation , 1985 .

[20]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[21]  James C. Anderson,et al.  Improper solutions in the analysis of covariance structures: Their interpretability and a comparison of alternate respecifications , 1987 .

[22]  Manabu Sato.  Pragmatic treatment of improper solutions in factor analysis , 1987 .

[23]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[24]  H. Akaike.  Factor analysis and AIC , 1987 .

[25]  Y. Kano,et al.  Identification of inconsistent variates in factor analysis , 1994 .

[26]  R. Gill,et al.  Conditions for factor (in)determinacy in factor analysis , 1998 .

[27]  Yutaka Kano,et al.  Improper Solutions in Exploratory Factor Analysis: Causes and Treatments , 1998 .

[28]  S. J. Press,et al.  A note on choosing the number of factors , 1999 .

[29]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[30]  K. Bollen,et al.  Improper Solutions in Structural Equation Models , 2001 .

[31]  Sik-Yum Lee,et al.  Bayesian Selection on the Number of Factors in a Factor Analysis Model , 2002 .

[32]  D. Flora,et al.  An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. , 2004, Psychological methods.

[33]  Michael A. West,et al.  BAYESIAN MODEL ASSESSMENT IN FACTOR ANALYSIS , 2004 .

[34]  S. Konishi,et al.  Bayesian information criteria and smoothing parameter selection in radial basis function networks , 2004 .

[35]  C. Anderson‐Cook,et al.  An Introduction to Multivariate Statistical Analysis (3rd ed.) (Book) , 2004 .

[36]  David B. Dunson,et al.  Efficient Bayesian Model Averaging in Factor Analysis , 2006 .

[37]  G. Kitagawa,et al.  Information Criteria and Statistical Modeling , 2007 .

[38]  Kei Hirose,et al.  Bayesian Factor Analysis and Model Selection , 2008 .

[39]  Ernest Fokoue,et al.  Bayesian Computation of the Intrinsic Structure of Factor Analytic Models , 2009, Journal of Data Science.