Identification of Items that Show Nonuniform DIF

This study compared three procedures—the Mantel- Haenszel (MH), the simultaneous item bias (SIB), and the logistic regression (LR) procedures—with respect to their Type I error rates and power to detect nonuniform dif ferential item functioning (DIF). Data were simulated to reflect a variety of conditions: The factors manipulated included sample size, ability distribution differences between the focal and the reference groups, proportion of DIF items in the test, DIF effect sizes, and type of item. 384 conditions were studied. Both the SIB and LR proce dures were equally powerful in detecting nonuniform DIF under most conditions. The MH procedure was not very effective in identifying nonuniform DIF items that had disordinal interactions. The Type I error rates were within the expected limits for the MH procedure and were higher than expected for the SIB and LR proce dures ; the SIB results showed an overall increase of approximately 1% over the LR results. Index terms: differential item functioning, logistic regression statistic, Mantel-Haenszel statistic, nondirectional DIF, simulta neous item bias statistic, SIBTEST, Type I error rate, unidirectional DIF.

[1]  Dorothy T. Thayer,et al.  Differential Item Performance and the Mantel-Haenszel Procedure. , 1986 .

[2]  Hariharan Swaminathan,et al.  Performance of the Mantel-Haenszel and Simultaneous Item Bias Procedures for Detecting Differential Item Functioning , 1993 .

[3]  William Stout,et al.  A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF , 1993 .

[4]  W. Stout,et al.  A new procedure for detection of crossing DIF , 1996 .

[5]  Neil J. Dorans,et al.  Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. , 1986 .

[6]  R. Berk.  Handbook of methods for detecting test bias , 1982 .

[7]  R. Hambleton,et al.  The Effect of Sample Size on the Functioning of the Mantel-Haenszel Statistic , 1992 .

[8]  James L. Wardrop,et al.  Item Bias in a Test of Reading Comprehension , 1981 .

[9]  An Investigation of Item Bias in a Test of Reading Comprehension. Technical Report No. 163. , 1980 .

[10]  G. J. Mellenbergh.  Contingency Table Models for Assessing Item Bias , 1982 .

[11]  Identifiers California,et al.  Annual Meeting of the National Council on Measurement in Education , 1998 .

[12]  H. Swaminathan,et al.  A Comparison of Logistic Regression and Mantel-Haenszel Procedures for Detecting Differential Item Functioning , 1993 .

[13]  Nambury S. Raju,et al.  The area between two item characteristic curves , 1988 .

[14]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[15]  H. Swaminathan,et al.  Detecting Differential Item Functioning Using Logistic Regression Procedures , 1990 .

[16]  R. D. Bock,et al.  Multivariate Statistical Methods in Behavioral Research , 1978 .

[17]  R. Hambleton,et al.  Detecting potentially biased test items : Comparison of IRT area and Mantel-Haenszel methods , 1989 .