ITEM BIAS INDICES BASED ON TOTAL TEST SCORE AND JOB PERFORMANCE ESTIMATES OF ABILITY

Researchers (e.g., Ironson, 1982; Tenopyr, 1990) have suggested that item bias investigators equate subgroups on external criteria such as job performance rather than total test scores before considering subgroup passing rates on test items. In a study comparing these two approaches to studies of item bias, we found little evidence of bias using total test score as the estimate of overall examinee ability, but nearly all items were biased in comparisons of white and African-American subgroups on Numerical, Verbal, and Mechanical Reasoning tests and in male-female comparisons on a Mechanical Reasoning test when job performance was used to select “equally able” examinees. However, the use of job performance as the ability index is analogous to performance-based approaches to test bias (Hartigan & Wigdor, 1989; Thorndike, 1971) and directly equivalent to the Darlington (1971) and Cole (1973) test bias definition, the logical inconsistencies of which have been previously described (Hunter & Schmidt, 1976; Peterson & Novick, 1976). We conclude that performance matching as a basis of forming “equal ability” groups is inappropriate.