Measurement Invariance, Predictive Invariance, and the Duality Paradox.

The statistical literature on bias in psychological testing distinguishes at least two forms of bias: measurement bias and predictive bias. Measurement bias concerns group differences in the relationship between a test and the latent variable to be measured. Predictive bias concerns group differences in the relationship between a test and an external criterion. How are these two forms of bias related? For example. if a test is unbiased in the predictive sense, does this fact support the hypothesis that the test is unbiased in the measurement sense? A theorem is given that describes the conditions under which measurement invariance (lack of bias) is consistent with predictive invariance for the linear case. Paradoxically, these two forms of invariance are shown to be inconsistent under realistic conditions. This duality or inconsistency is illustrated in simulated data. The implications of the duality for group differences research are illustrated in real data involving gender and ethnic differences on the SAT. The phenomenon of duality may force a reinterpretation of common empirical findings of test criterion regression slope invariance. and of invariance in test validities. Other implications are discussed.

[1]  D. Geary,et al.  Father's Occupation and Social Background: Relation to SAT Scores , 1984 .

[2]  A. Jensen,et al.  Spearman's Hypothesis: Methodology and Evidence. , 1992, Multivariate behavioral research.

[3]  K. Kraiger,et al.  Study of race effects in objective indices and subjective evaluations of performance: A meta-analysis of performance criteria. , 1986 .

[4]  Alieia P. Sehmitt,et al.  Differential Item Functioning for Minority Examinees on the SAT , 1990 .

[5]  T. Cleary.  TEST BIAS: PREDICTION OF GRADES OF NEGRO AND WHITE STUDENTS IN INTEGRATED COLLEGES , 1968 .

[6]  R E Millsap,et al.  Statistical Evidence in Salary Discrimination Studies: Nonparametric Inferential Conditions. , 1994, Multivariate behavioral research.

[7]  R. P. McDonald,et al.  Choosing a multivariate model: Noncentrality and goodness of fit. , 1990 .

[8]  Gideon J. Mellenbergh,et al.  Item bias and item response theory , 1989 .

[9]  Gerald V. Barrett,et al.  Validity of Personnel Decisions: A Conceptual Analysis of the Inferential and Evidential Bases , 1989 .

[10]  L. Gottfredson.  Reconsidering fairness: A matter of social and ethical priorities , 1988 .

[11]  Malcolm James Ree,et al.  Predicting job performance: Not much more than g.. , 1994 .

[12]  Dag Sörbom,et al.  An alternative to the methodology for analysis of covariance , 1978 .

[13]  Kurt Kraiger,et al.  A meta-analysis of ratee race effects in performance ratings. , 1985 .

[14]  F. Schmidt.  The Problem of Group Differences in Ability Test Scores in Employment Selection , 1988 .

[15]  Jerard F. Kehoe,et al.  On the fair use of bias: A comment on Drasgow. , 1983 .

[16]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[17]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[18]  L. V. Jones.  The Influence on Mathematics Test Scores, by Ethnicity and Sex, of Prior Achievement and High School Mathematics Courses. , 1987 .

[19]  Roger E. Millsap,et al.  On the misuse of manifest variables in the detection of measurement bias , 1992 .

[20]  Steffanie L. Wilk,et al.  Within-group norming and other forms of score adjustment in preemployment testing. , 1994, The American psychologist.

[21]  F. Schmidt,et al.  Methodological, Statistical, and Ethical Issues in the Study of Bias in Psychological Tests , 1984 .

[22]  A. Goldberger.  Reverse Regression and Salary Discrimination , 1984 .

[23]  R. Linn.  SELECTION BIAS: MULTIPLE MEANINGS , 1984 .

[24]  Fritz Drasgow,et al.  Biased test items and differential validity , 1982 .

[25]  J. H. Steiger.  Structural Model Evaluation and Modification: An Interval Estimation Approach. , 1990, Multivariate behavioral research.

[26]  L. Gottfredson,et al.  Validity versus Utility of Mental Tests: Example of the SAT. , 1986 .

[27]  J. Hunter.  Cognitive ability, cognitive aptitudes, job knowledge, and job performance , 1986 .