Finite mixtures in confirmatory factor-analysis models

In this paper, various types of finite mixtures of confirmatory factor-analysis models are proposed for handling data heterogeneity. Under the proposed mixture approach, observations are assumed to be drawn from mixtures of distinct confirmatory factor-analysis models. But each observation does not need to be identified to a particular model prior to model fitting. Several classes of mixture models are proposed. These models differ by their unique representations of data heterogeneity. Three different sampling schemes for these mixture models are distinguished. A mixed type of the these three sampling schemes is considered throughout this article. The proposed mixture approach reduces to regular multiple-group confirmatory factor-analysis under a restrictive sampling scheme, in which the structural equation model for each observation is assumed to be known. By assuming a mixture of multivariate normals for the data, maximum likelihood estimation using the EM (Expectation-Maximization) algorithm and the AS (Approximate-Scoring) method are developed, respectively. Some mixture models were fitted to a real data set for illustrating the application of the theory. Although the EM algorithm and the AS method gave similar sets of parameter estimates, the AS method was found computationally more efficient than the EM algorithm. Some comments on applying the mixture approach to structural equation modeling are made.

[1]  S. John,et al.  On Identifying the Population of Origin of Each Observation in a Mixture of Observations from Two Normal Populations , 1970 .

[2]  C. Robertson,et al.  A comparison of some methods for estimating mixed normal distributions , 1972 .

[3]  R. Sundberg.  An iterative method for solution of the likelihood equations for incomplete data from exponential families , 1976 .

[4]  K. Holzinger,et al.  A study in factor analysis : the stability of a bi-factor solution , 1939 .

[5]  Keewhan Choi,et al.  An Estimation Procedure for Mixtures of Distributions , 1968 .

[6]  Bhattacharya Cg.  A simple method of resolution of a distribution into gaussian components. , 1967 .

[7]  Studies in Nonlinear Estimation. , 1977 .

[8]  M. Aitkin,et al.  Mixture Models, Outliers, and the EM Algorithm , 1980 .

[9]  W. Y. Tan,et al.  Some Comparisons of the Method of Moments and the Method of Maximum Likelihood in Estimating Parameters of a Mixture of Two Normal Densities , 1972 .

[10]  Calyampudi R. Rao,et al.  Advanced Statistical Methods in Biometric Research. , 1953 .

[11]  Michael J. Symons,et al.  Clustering criteria and multivariate normal mixtures , 1981 .

[12]  H. Teicher.  Identifiability of Mixtures , 1961 .

[13]  M. Rajagopalan,et al.  Bayes estimates of mixing proportions in finite mixture distributions , 1991 .

[14]  Keewhan Choi.  Estimators for the parameters of a finite mixture of distributions , 1969 .

[15]  Javad Behboodian,et al.  On a mixture of normal distributions , 1970 .

[16]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics (Revised Edition) , 1999 .

[17]  D. Titterington.  Some recent research in the analysis of mixture distributions , 1990 .

[18]  Peter M. Bentler,et al.  Covariance structure analysis with heterogeneous kurtosis parameters , 1990 .

[19]  A. Scott,et al.  Clustering methods based on likelihood ratio criteria. , 1971 .

[20]  G. J. McLachlan,et al.  9 The classification and mixture maximum likelihood approaches to cluster analysis , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[21]  N. W. Please.  COMPARISON OF FACTOR LOADINGS IN DIFFERENT POPULATIONS , 1973 .

[22]  E. Lehmann.  Efficient Likelihood Estimators , 1980 .

[23]  Geoffrey J. McLachlan,et al.  Some Efficiency Results for the Estimation of the Mixing Proportion in a Mixture of 2 Normal-Distributions , 1981 .

[24]  D. Sörbom.  A GENERAL METHOD FOR STUDYING DIFFERENCES IN FACTOR MEANS AND FACTOR STRUCTURE BETWEEN GROUPS , 1974 .

[25]  Michael J. Hartley,et al.  Estimating Mixtures of Normal Distributions and Switching Regressions: Comment , 1978 .

[26]  Patrick L. Odell,et al.  Concerning several methods for estimating crop acreages using remote sensing data , 1976 .

[27]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[28]  B. Everitt,et al.  Finite Mixture Distributions , 1981 .

[29]  M. Degroot,et al.  Modeling lake-chemistry distributions: approximate Bayesian methods for estimating a finite-mixture model , 1992 .

[30]  J. Wolfe.  PATTERN CLUSTERING BY MULTIVARIATE MIXTURE ANALYSIS. , 1970, Multivariate behavioral research.

[31]  J. B. Ramsey,et al.  Estimating Mixtures of Normal Distributions and Switching Regressions , 1978 .

[32]  N. Kiefer.  Discrete Parameter Variation: Efficient Estimation of a Switching Regression Model , 1978 .

[33]  S. Yakowitz,et al.  On the Identifiability of Finite Mixtures , 1968 .

[34]  S. Sclove.  Population mixture models and clustering algorithms , 1977 .

[35]  Geoffrey J. McLachlan,et al.  Estimation of Mixing Proportions: A Case Study , 1984 .

[36]  B. Lindsay,et al.  Multivariate Normal Mixtures: A Fast Consistent Method of Moments , 1993 .

[37]  H. Teicher.  Identifiability of Finite Mixtures , 1963 .

[38]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[39]  David David.  Maximum likelihood estimates of the parameters of a mixture of two regression lines , 1974 .

[40]  Michael W. Browne,et al.  Topics in Applied Multivariate Analysis: COVARIANCE STRUCTURES , 1982 .

[41]  S. Yakowitz.  A Consistent Estimator for the Identification of Finite Mixtures , 1969 .

[42]  H. Teicher.  On the Mixture of Distributions , 1960 .

[43]  V. Hasselblad.  Estimation of parameters for a mixture of normal distributions , 1966 .

[44]  P. Bentler,et al.  Multiple population covariance structure analysis under arbitrary distribution theory , 1987 .

[45]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[46]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[47]  A. F. Smith,et al.  A Quasi‐Bayes Sequential Procedure for Mixtures , 1978 .

[48]  Kenneth A. Bollen,et al.  Structural Equations with Latent Variables , 1989 .

[49]  R. Hathaway.  A constrained EM algorithm for univariate normal mixtures , 1986 .

[50]  K. Jöreskog.  Simultaneous factor analysis in several populations , 1971 .

[51]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[52]  J. Hartigan.  Distribution Problems in Clustering , 1977 .

[53]  R. Quandt.  A New Approach to Estimating Switching Regressions , 1972 .

[54]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[55]  V. Hasselblad.  Finite mixtures of distributions from the exponential family , 1969 .

[56]  Ronald Schoenberg,et al.  Application of the EM Method , 1984 .

[57]  D. Hosmer,et al.  Information and mixtures of two normal distributions , 1977 .

[58]  The Chi-Squared Distribution , 1971 .

[59]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .

[60]  Calyampudi R. Rao,et al.  Advanced Statistical Methods in Biometric Research. , 1953 .

[61]  Sik-Yum Lee,et al.  Covariance structure analysis in several populations , 1982 .

[62]  N. E. Day.  Estimating the components of a mixture of normal distributions , 1969 .

[63]  B. Lindsay.  Moment Matrices: Applications in Mixtures , 1989 .

[64]  B. Muthén.  Latent variable modeling in heterogeneous populations , 1989 .

[65]  B. Lindsay,et al.  Testing for the number of components in a mixture of normal distributions using moment estimators , 1994 .

[66]  R. Hathaway.  A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions , 1985 .

[67]  B. Lindsay,et al.  Measuring the relative effectiveness of moment estimators as starting values in maximizing likelihoods , 1994 .

[68]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .