Power and Precision in Confirmatory Factor Analytic Tests of Measurement Invariance

This study investigates the effects of sample size, factor overdetermination, and communality on the precision of factor loading estimates and the power of the likelihood ratio test of factorial invariance in multigroup confirmatory factor analysis. Although sample sizes are typically thought to be the primary determinant of precision and power, the degree of factor overdetermination and the level of indicator communalities also play important roles. Based on these findings, no single rule of thumb regarding the ratio of sample size to number of indicators can ensure adequate power to detect a lack of measurement invariance.

[1]  Kristine Y. Hogarty,et al.  The Quality of Factor Solutions in Exploratory Factor Analysis: The Influence of Sample Size, Communality, and Overdetermination , 2005 .

[2]  N. Schmitt,et al.  Interindividual differences in intraindividual changes in proactivity during organizational entry: a latent growth modeling approach to understanding newcomer adaptation. , 2000, The Journal of applied psychology.

[3]  F. Drasgow.  Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are the central issues , 1984 .

[4]  James C. Anderson,et al.  Improper solutions in the analysis of covariance structures: Their interpretability and a comparison of alternate respecifications , 1987 .

[5]  Roger E. Millsap,et al.  Invariance in measurement and prediction: Their relationship in the single-factor case. , 1997 .

[6]  R. Millsap,et al.  Evaluating the impact of partial factorial invariance on selection in two populations. , 2004, Psychological methods.

[7]  H. Marsh.  The Factorial Invariance of Responses by Males and Females to a Multidimensional Self-Concept Instrument: Substantive and Methodological Issues. , 1987, Multivariate behavioral research.

[8]  R. Vandenberg.  Toward a Further Understanding of and Improvement in Measurement Invariance Methods and Procedures , 2002 .

[9]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[10]  N. Schmitt,et al.  Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions. , 1997, The Journal of applied psychology.

[11]  Robert E. Ployhart,et al.  WEB‐BASED AND PAPER‐AND‐PENCIL TESTING OF APPLICANTS IN A PROCTORED SETTING: ARE PERSONALITY, BIODATA, AND SITUATIONAL JUDGMENT TESTS COMPARABLE? , 2003 .

[12]  David Chan,et al.  The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM) , 1998 .

[13]  R. MacCallum,et al.  Sample size in factor analysis. , 1999 .

[14]  Robert J. Vandenberg,et al.  A Central Question in Cross-Cultural Research: Do Employees of Different Cultures Interpret Work-related Measures in an Equivalent Manner?: , 1994 .

[15]  J. Hair.  Multivariate data analysis , 1972 .

[16]  Robert E. Ployhart,et al.  Applications of Mean and Covariance Structure Analysis: Integrating Correlational and Experimental Approaches , 2004 .

[17]  E. Kevin Kelloway,et al.  Structural equation modelling in perspective. , 1995 .

[18]  H. Marsh,et al.  Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. , 1985 .

[19]  H W Marsh,et al.  Is More Ever Too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis. , 1998, Multivariate behavioral research.

[20]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[21]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[22]  D. Kaplan.  Power of the Likelihood Ratio Test in Multiple Group Confirmatory Factor Analysis under Partial Measurement Invariance , 1989 .

[23]  Fritz Drasgow,et al.  Study of the measurement bias of two standardized psychological tests. , 1987 .

[24]  Adam W. Meade,et al.  A Monte-Carlo Study of Confirmatory Factor Analytic Tests of Measurement Equivalence/Invariance , 2004 .

[25]  H. Marsh.  The Structure of Masculinity/Femininity: An Application of Confirmatory Factor Analysis to Higher-Order Factor Structures and Factorial Invariance. , 1985, Multivariate behavioral research.

[26]  S. Craig,et al.  Are performance appraisal ratings from different rating sources comparable? , 2001, The Journal of applied psychology.

[27]  R E Millsap,et al.  Measurement Invariance, Predictive Invariance, and the Duality Paradox. , 1995, Multivariate behavioral research.

[28]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[29]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[30]  Adam W. Meade,et al.  Problems With Item Parceling for Confirmatory Factor Analytic Tests of Measurement Invariance , 2006 .

[31]  H. Wainer,et al.  Differential Item Functioning. , 1994 .

[32]  Michael T. Brannick,et al.  Critical comments on applying covariance structure modeling , 1995 .

[33]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[34]  Roger E. Millsap,et al.  On the misuse of manifest variables in the detection of measurement bias , 1992 .