Factors causing occurrence of artificial dif: A simulation study for dichotomous data

ABSTRACT In modern test theory, differential item functioning (DIF) appears where respondents from two different groups have the same ability but different probability to respond to an item correctly. If some items favouring one group lead to the appearance of DIF in any other item favouring the other group, this type of problem is called artificial differential item functioning (A-DIF). The purpose of this paper is to deal with the effect of different factors causing A-DIF under the Rasch theoretical model for dichotomous responses. A simulation study was conducted to explore how various factors affect real DIF and simultaneous A-DIF expected proportion including total sample size, percentage of individuals in focal and reference groups, percentage of items exhibiting real DIF and DIF magnitude for two item sets including 10 and 20 items. It is concluded that DIF magnitude is the most essential factor while evaluating A-DIF for each item set. This is followed by percent of items exhibiting real-DIF.

[1]  D. Andrich,et al.  Real and Artificial Differential Item Functioning in Polytomous Items , 2015, Educational and psychological measurement.

[2]  Cynthia G. Parshall,et al.  Exact Versus Asymptotic Mantel-Haenszel DIF Statistics: A Comparison of Performance Under Small-Sample Conditions , 1995 .

[3]  W. Haenszel,et al.  Statistical aspects of the analysis of data from retrospective studies of disease. , 1959, Journal of the National Cancer Institute.

[4]  W. Meredith,et al.  Inferential Conditions in the Statistical Detection of Measurement Bias , 1992 .

[5]  C. Walker.  What’s the DIF? Why Differential Item Functioning Analyses Are an Important Part of Instrument Development and Validation , 2011 .

[6]  Adam E. Wyse,et al.  Differential Item Functioning Analysis Using Rasch Item Information Functions , 2009 .

[7]  Roger E. Millsap,et al.  On the misuse of manifest variables in the detection of measurement bias , 1992 .

[8]  L. Shepard,et al.  Methods for Identifying Biased Test Items , 1994 .

[9]  R. Zwick.  When Do Item Response Function and Mantel-Haenszel Definitions of Differential Item Functioning Coincide? , 1990 .

[10]  C. Hagquist.  Determinants of Artificial DIF : a study based on simulated polytomous data , 2015 .

[11]  Dorothy T. Thayer,et al.  Differential Item Performance and the Mantel-Haenszel Procedure. , 1986 .

[12]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[13]  Brian E. Clauser,et al.  Using Statistical Procedures to Identify Differentially Functioning Test Items , 2005 .

[14]  A Matter of Test Bias in Educational Policy Research: Bringing the Context into Picture by Investigating Sociological/Community Moderated (or Mediated) Test and Item Bias. , 2005 .

[15]  Curt Hagquist,et al.  Real and Artificial Differential Item Functioning , 2012 .

[16]  B. Zumbo.  A Handbook on the Theory and Methods of Differential Item Functioning (DIF) LOGISTIC REGRESSION MODELING AS A UNITARY FRAMEWORK FOR BINARY AND LIKERT-TYPE (ORDINAL) ITEM SCORES , 1999 .