A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research

The establishment of measurement invariance across groups is a logical prerequisite to conducting substantive cross-group comparisons (e.g., tests of group mean differences, invariance of structural parameter estimates), but measurement invariance is rarely tested in organizational research. In this article, the authors (a) elaborate the importance of conducting tests of measurement invariance across groups, (b) review recommended practices for conducting tests of measurement invariance, (c) review applications of measurement invariance tests in substantive applications, (d) discuss issues involved in tests of various aspects of measurement invariance, (e) present an empirical example of the analysis of longitudinal measurement invariance, and (f) propose an integrative paradigm for conducting sequences of measurement invariance tests.

[1]  Harold Gulliksen,et al.  Regression tests for several samples , 1950, Psychometrika.

[2]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[3]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[4]  William Meredith,et al.  Rotation to achieve factorial invariance , 1964 .

[5]  William Meredith,et al.  Notes on factorial invariance , 1964 .

[6]  R. C. Durfee,et al.  MULTIPLE FACTOR ANALYSIS. , 1967 .

[7]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[8]  K. Jöreskog.  A general approach to confirmatory maximum likelihood factor analysis , 1969 .

[9]  L. Cronbach,et al.  How we should measure "change": Or should we? , 1970 .

[10]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[11]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[12]  Robert T. Golembiewski,et al.  Measuring Change and Persistence in Human Affairs: Types of Change Generated by OD Designs , 1976 .

[13]  C. Werts,et al.  The Use Of Analysis Of Covariance Structures For Comparing The Psychometric Properties Of Multiple Variables Across Populations. , 1978, Multivariate behavioral research.

[14]  Robert W. Zmud,et al.  INTERPRETING THE MEASUREMENT OF CHANGE IN ORGANIZATIONAL RESEARCH1 , 1979 .

[15]  Scott E. Maxwell,et al.  Evaluating Planned Organizational Change: A Method for Assessing Alpha, Beta, and Gamma Change , 1980 .

[16]  Ability Factor Structure Differences In Adulthood And Old Age. , 1981, Multivariate behavioral research.

[17]  N. Schmitt,et al.  The Use Of Analysis Of Covariance Structures To Assess Beta And Gamma Change. , 1982, Multivariate behavioral research.

[18]  S. Whitely.  Construct validity: Construct representation versus nomothetic span. , 1983 .

[19]  R. Brennan.  Elements of generalizability theory , 1983 .

[20]  Neal Schmitt,et al.  Comparison of Three Techniques to Assess Group-Level Beta and Gamma Change , 1984 .

[21]  F. Drasgow.  Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are the central issues , 1984 .

[22]  R. Kanfer,et al.  Equivalence of psychological measurement in heterogeneous populations. , 1985, The Journal of applied psychology.

[23]  H. Marsh,et al.  Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. , 1985 .

[24]  Scott E. Maxwell,et al.  Multitrait-Multimethod Comparisons Across Populations: A Confirmatory Factor Analytic Approach. , 1985, Multivariate behavioral research.

[25]  Karl G. Jöreskog,et al.  Simultaneous Analysis of Longitudinal Data From Several Cohorts , 1985 .

[26]  C. Hertzog,et al.  Measurement in the psychology of adulthood and aging. , 1985 .

[27]  A. Shapiro,et al.  On the multivariate asymptotic distribution of sequential Chi-square statistics , 1985 .

[28]  Jennifer A. Chatman,et al.  Organizational commitment and psychological attachment: The effects of compliance, identification, and internalization on prosocial behavior. , 1986 .

[29]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[30]  C. Hertzog,et al.  Stability and change in adult intelligence: 1. Analysis of longitudinal covariance structures. , 1986, Psychology and aging.

[31]  Barbara M. Byrne,et al.  Adolescent Self-Concept: Testing the Assumption of Equivalent Structure Across Gender , 1987 .

[32]  H. Marsh.  The Factorial Invariance of Responses by Males and Females to a Multidimensional Self-Concept Instrument: Substantive and Methodological Issues. , 1987, Multivariate behavioral research.

[33]  Fritz Drasgow,et al.  Study of the measurement bias of two standardized psychological tests. , 1987 .

[34]  Jean M. Bartunek,et al.  The Effects of Organizational Restructuring on Frames of Reference and Cooperation , 1988 .

[35]  F. W. Vondracek,et al.  Factorial Invariance of Career Indecision Dimensions across Junior High and High School Males and Females. , 1988 .

[36]  Sandra B. Hartog,et al.  Alpha, beta, and gamma change in evaluation research: A structural equation approach. , 1988 .

[37]  James C. Anderson,et al.  STRUCTURAL EQUATION MODELING IN PRACTICE: A REVIEW AND RECOMMENDED TWO-STEP APPROACH , 1988 .

[38]  S. Iwawaki,et al.  Cross-Cultural Comparability of Temperament Among Japanese and American Preschool Children , 1988 .

[39]  Measuring Adolescent Self-Concept: Factorial Validity and Equivalency of the SDQ III Across Gender. , 1988, Multivariate behavioral research.

[40]  R. P. McDonald,et al.  Goodness-of-fit indexes in confirmatory factor analysis : The effect of sample size , 1988 .

[41]  J. Mcardle.  Dynamic but Structural Equation Modeling of Repeated Measures Data , 1988 .

[42]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[43]  Stephen G. Green,et al.  Confirmatory factor analytic procedures for assessing change during organizational entry , 1989 .

[44]  J. Finch,et al.  Positive and negative social ties among older adults: Measurement models and the prediction of psychological distress and well-being , 1989, American journal of community psychology.

[45]  M J Burke,et al.  Measuring affect at work: confirmatory analyses of competing mood structures with conceptual linkage to cortical regulatory systems. , 1989, Journal of personality and social psychology.

[46]  S. Mulaik,et al.  EVALUATION OF GOODNESS-OF-FIT INDICES FOR STRUCTURAL EQUATION MODELS , 1989 .

[47]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[48]  B. Byrne.  A Primer of Lisrel: Basic Applications and Programming for Confirmatory Factor Analytic Models , 1989 .

[49]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[50]  Vida Scarpello,et al.  The matching model: An examination of the processes underlying realistic job previews. , 1990 .

[51]  R. P. McDonald,et al.  Choosing a multivariate model: Noncentrality and goodness of fit. , 1990 .

[52]  J. Mcardle,et al.  Patterns of Change within Latent Variable Structural Equation Models , 1990 .

[53]  J. H. Steiger.  Structural Model Evaluation and Modification: An Interval Estimation Approach. , 1990, Multivariate behavioral research.

[54]  R E Millsap,et al.  Confirmatory Measurement Model Comparisons Using Latent Means. , 1991, Multivariate behavioral research.

[55]  B. Byrne.  The Maslach Burnout Inventory: Validating Factorial Structure and Invariance Across Intermediate, Secondary, and University Educators. , 1991, Multivariate behavioral research.

[56]  P. Prescott,et al.  Patient intensity for nursing index: the measurement model. , 1991, Research in nursing & health.

[57]  Richard P. Bagozzi,et al.  Assessing Construct Validity in Organizational Research , 1991 .

[58]  Gary R. Pike,et al.  Using structural equation models with latent variables to study student growth and development , 1991 .

[59]  J. Tisak,et al.  Psychometric equivalence of a translated circadian rhythm questionnaire: implications for between- and within-population assessments. , 1991, The Journal of applied psychology.

[60]  R. Lennox,et al.  Conventional wisdom on measurement: A structural equation perspective. , 1991 .

[61]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[62]  R. Landis,et al.  Equivalence of constructs measured by job-specific and commercially available aptitude tests , 1992 .

[63]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[64]  Richard G. Netemeyer,et al.  Assessing the Cross-National Applicability of Consumer Behavior Models: A Model of Attitude toward Advertising in General , 1993 .

[65]  Herbert W. Marsh,et al.  The Multidimensional Structure of Academic Self-Concept: Invariance Over Gender and Age , 1993 .

[66]  B. Byrne,et al.  Confirmatory Factor Analysis of Multitrait-Multimethod Self-concept Data: Between-group and Within-group Invariance Constraints. , 1993, Multivariate behavioral research.

[67]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[68]  C. R. Snyder,et al.  Psychometric Properties of the Hope Scale: A Confirmatory Factor Analysis , 1993 .

[69]  Robert J. Vandenberg,et al.  Assessing newcomers' changing commitments to the organization during the first 6 months of work , 1993 .

[70]  B. Byrne,et al.  Measuring Adolescent Depression: Factorial Validity and Invariance of the Beck Depression Inventory Across Gender , 1993 .

[71]  D. Mackinnon,et al.  Generality and specificity in health behavior: application to warning-label and social influence expectancies. , 1993, The Journal of applied psychology.

[72]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[73]  Michael A. Campion,et al.  ARTICLE REVIEW CHECKLIST: A CRITERION CHECKLIST FOR REVIEWING RESEARCH ARTICLES IN APPLIED PSYCHOLOGY , 1993 .

[74]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[75]  Scott E. Maxwell,et al.  Multivariate group comparisons of variable systems: MANOVA and structural equation modeling. , 1993 .

[76]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[77]  B. Byrne,et al.  The Beck Depression Inventory: Testing and Cross-Validating a Hierarchical Factor Structure for Nonclinical Adolescents. , 1993 .

[78]  H. Marsh.  Confirmatory factor analysis models of factorial invariance: A multifaceted approach , 1994 .

[79]  Robert J. Vandenberg,et al.  A Central Question in Cross-Cultural Research: Do Employees of Different Cultures Interpret Work-related Measures in an Equivalent Manner?: , 1994 .

[80]  John B. Willett,et al.  Using covariance structure analysis to detect correlates and predictors of individual change over time , 1994 .

[81]  B. Byrne,et al.  Measuring Adolescent Depression: Tests of Equivalent Factorial Structure for English and French Versions of the Beck Depression Inventory , 1994 .

[82]  M A Pentz,et al.  Measurement invariance in longitudinal clinical research assuming change from development and intervention. , 1994, Journal of consulting and clinical psychology.

[83]  B. Byrne,et al.  Testing for the Factorial Validity, Replication, and Invariance of a Measuring Instrument: A Paradigmatic Application Based on the Maslach Burnout Inventory. , 1994, Multivariate behavioral research.

[84]  Gina J. Medsker,et al.  A Review of Current Practices for Evaluating Causal Models in Organizational Behavior and Human Resources Management Research , 1994 .

[85]  Terry E. Duncan,et al.  Modeling Developmental Processes Using Latent Growth Structural Equation Methodology , 1994 .

[86]  F. Vijver,et al.  The incomplete equivalence of the paper-and-pencil and computerized versions of the General Aptitude Test Battery , 1994 .

[87]  R. Vandenberg,et al.  A Critical Examination of the Internalization, Identification, and Compliance Commitment Measures , 1994 .

[88]  R. Bagozzi,et al.  A general approach to representing multifaceted personality constructs: Application to state self‐esteem , 1994 .

[89]  B. Byrne,et al.  The Beck Depression Inventory (French Version): Testing for Gender-Invariant Factorial Structure for Nonclinical Adolescents , 1994 .

[90]  J. Nesselroade,et al.  SELECTION AND RELATED THREATS TO GROUP COMPARISONS: AN EXAMPLE COMPARING FACTORIAL STRUCTURES OF HIGHER AND LOWER ABILITY GROUPS OF ADULT TWINS , 1995 .

[91]  Edward W. Miles,et al.  A quasi-experimental assessment of the effect of computerizing noncognitive paper-and-pencil measurements: A test of measurement equivalence. , 1995 .

[92]  Rodger W. Griffeth,et al.  Managing in the International Context: Testing Cultural Generality of Sources of Commitment to Multinational Enterprises , 1995 .

[93]  M. Schmit,et al.  Frame-of-reference effects on personality scale scores and criterion-related validity. , 1995 .

[94]  Terry E. Duncan,et al.  Modeling the processes of development via latent variable growth curve methodology , 1995 .

[95]  H. Marsh.  Δ2 and χ2 I2 fit indices for structural equation models: A brief note of clarification , 1995 .

[96]  Timothy R. Hinkin.  A Review of Scale Development Practices in the Study of Organizations , 1995 .

[97]  Gary R. Pike,et al.  Limitations of using students' self-reports of academic development as proxies for traditional achievement measures , 1995 .

[98]  Jagdip Singh,et al.  Measurement Issues in Cross-National Research , 1995 .

[99]  J. Hittner.  Factorial validity and equivalency of the Alcohol Expectancy Questionnaire Tension-Reduction Subscale across gender and drinking frequency. , 1995, Journal of clinical psychology.

[100]  Eunsook Hong,et al.  A structural comparison between state and trait self‐regulation models , 1995 .

[101]  Michael T. Brannick,et al.  Critical comments on applying covariance structure modeling , 1995 .

[102]  C. Ruetsch,et al.  Testing for Equivalence of Measurement Scales: Simple Structure and Metric Invariance Reconsidered. , 1995, Multivariate behavioral research.

[103]  E. Kevin Kelloway,et al.  Structural equation modelling in perspective. , 1995 .

[104]  J. Nesselroade,et al.  Selection and related threats to group comparisons: an example comparing factorial structures of higher and lower ability groups of adult twins. , 1995, Psychological bulletin.

[105]  H. Marsh,et al.  An evaluation of incremental fit indices: A clarification of mathematical and empirical properties. , 1996 .

[106]  P. Treffers,et al.  Measuring children's self-concept with a Dutch version of the "self-perception profile for children": factorial validity and invariance across a nonclinic and a clinic group. , 1996, Journal of personality assessment.

[107]  P. J. Ferrando.  Calibration of Invariant Item Parameters in a Continuous Item Response Model Using the Extended Lisrel Measurement Submodel. , 1996, Multivariate behavioral research.

[108]  H. Marsh,et al.  Structure of artistic self-concepts for performing arts and non-performing arts students in a performing arts high school: "Setting the stage" with multigroup confirmatory factor analysis. , 1996 .

[109]  G. A. Marcoulides,et al.  Advanced structural equation modeling : issues and techniques , 1996 .

[110]  F. Schmidt,et al.  Measurement Error in Psychological Research: Lessons From 26 Research Scenarios , 1996 .

[111]  P. Harmer,et al.  Cross-cultural validation of the task and ego orientation in sport questionnaire , 1996 .

[112]  Jenn-Yun Tein,et al.  Longitudinal measurement models in evaluation research: Examining stability and change , 1996 .

[113]  George A. Marcoulides,et al.  Estimating Variance Components in Generalizability Theory: The Covariance Structure Analysis Approach. Teacher's Corner. , 1996 .

[114]  David Chan,et al.  Cognitive Misfit of Problem-Solving Style at Work: A Facet of Person-Organization Fit , 1996 .

[115]  Mark W. Roosa,et al.  Examination of the Cross-cultural and Cross-language Equivalence of the Parenting Self-Agency Measure. , 1996 .

[116]  S. Roesch,et al.  A comparison of the factor structure of processing speed for younger and older adults: testing the assumption of measurement equivalence across age groups. , 1997, Psychology and aging.

[117]  D. Chan,et al.  Racial subgroup differences in predictive validity perceptions on personality and cognitive ability tests. , 1997, Journal of Applied Psychology.

[118]  Testing the cross-cultural validity of TEOSQ and its factor covariance and mean structures across gender. , 1997 .

[119]  Condom use in unmarried Latino men: a test of cultural constructs. , 1997, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[120]  T. Little.  Mean and Covariance Structures (MACS) Analyses of Cross-Cultural Data: Practical and Theoretical Issues. , 1997, Multivariate behavioral research.

[121]  Christopher P. Parker,et al.  Support for affirmative action, justice perceptions, and work attitudes: a study of gender and racial-ethnic group differences. , 1997, The Journal of applied psychology.

[122]  N. Schmitt,et al.  Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions. , 1997, The Journal of applied psychology.

[123]  J. Nijenhuis,et al.  Comparability of GATB scores for immigrants and majority group members: Some Dutch findings. , 1997 .

[124]  Aron M. Levin,et al.  A Generation X Scale: Creation and Validation , 1997 .

[125]  A. Marcoen,et al.  Meaning in life of young and elderly adults: An examination of the factorial validity and invariance of the Life Regard Index , 1997 .

[126]  Jeffrey A. Miller,et al.  The Invariance of the Self-Concept Construct across White and Hispanic Student Populations , 1997 .

[127]  John L. Horn,et al.  A robust five-factor structure of the 16PF : Strong evidence from independent rotation and confirmatory factorial invariance procedures , 1997 .

[128]  S. West,et al.  The investigation of personality structure: Statistical models , 1997 .

[129]  B. V. D. Van den Bergh,et al.  Self-concept in children: equivalence of measurement and structure across gender and grade of Harter's Self-Perception Profile for Children. , 1998, Journal of personality assessment.

[130]  David Chan,et al.  The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM) , 1998 .

[131]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[132]  N. Schmitt,et al.  Understanding pretest and posttest reactions to cognitive ability and personality tests. , 1998, The Journal of applied psychology.

[133]  K Warner Schaie,et al.  Longitudinal invariance of adult psychometric ability factor structures across 7 years. , 1998, Psychology and aging.

[134]  Judith M. Collins,et al.  Race, job applicants, and the Five-Factor Model of Personality: Implications for Black psychology, industrial/organizational psychology, and the Five-Factor Theory. , 1998 .

[135]  L. V. Dyne,et al.  Helping and Voice Extra-Role Behaviors: Evidence of Construct and Predictive Validity , 1998 .

[136]  P. Bentler,et al.  Fit indices in covariance structure modeling : Sensitivity to underparameterized model misspecification , 1998 .

[137]  Nambury S. Raju,et al.  Peer and subordinate performance appraisal measurement equivalence. , 1998 .

[138]  T. Taris,et al.  Assessing Stability and Change of Psychometric Properties of Multi-Item Concepts Across Different Situations: A General Approach , 1998 .

[139]  R. Bagozzi,et al.  A General Approach for Representing Constructs in Organizational Research , 1998 .

[140]  Dawn M. Gondoli,et al.  Structure and validity of parent and teacher perceptions of children's competence: A multitrait–multimethod–multigroup investigation. , 1998 .

[141]  Timothy R. Hinkin,et al.  A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires , 1998 .

[142]  Gordon W. Cheung,et al.  Testing Factorial Invariance across Groups: A Reconceptualization and Proposed New Method , 1999 .

[143]  David Chan,et al.  EMPLOYEE ATTITUDE SURVEYS IN A MULTINATIONAL ORGANIZATION: CONSIDERING LANGUAGE AND CULTURE IN ASSESSING MEASUREMENT EQUIVALENCE , 1999 .

[144]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[145]  Lance,et al.  Latent Growth Models of Individual Change: The Case of Newcomer Adjustment. , 2000, Organizational behavior and human decision processes.

[146]  Rick H. Hoyle,et al.  Confirmatory Factor Analysis , 1983 .

[147]  Gordon W. Cheung,et al.  The Effects of Model Parsimony and Sampling Error on the Fit of Structural Equation Models , 2001 .

[148]  Adam W. Meade,et al.  We Should Measure Change-and Here’s How , 2002 .