Distinguishing between Net and Global DIF in Polytomous Items

In this article, I address two competing conceptions of differential item functioning (DIF) in polytomously scored items. The first conception, referred to as net DIF, concerns between-group differences in the conditional expected value of the polytomous response variable. The second conception, referred to as global DIF, concerns the conditional dependence of group membership and the polytomous response variable. The distinction between net and global DIF is important because different DIF evaluation methods are appropriate for net and global DIF; no currently available method is universally the best for detecting both net and global DIF. Net and global DIF definitions are presented under two different, yet compatible, modeling frameworks: a traditional item response theory (IRT) framework, and a differential step functioning (DSF) framework. The theoretical relationship between the IRT and DSF frameworks is presented. Available methods for evaluating net and global DIF are described, and an applied example of net and global DIF is presented.

[1]  Randall D. Penfield.  An Approach for Categorizing DIF in Polytomous Items , 2007 .

[2]  Tony C. M. Lam,et al.  Assessing Differential Item Functioning in Performance Assessment: Review and Recommendations. , 2005 .

[3]  Nathan Mantel,et al.  Chi-square tests with one degree of freedom , 1963 .

[4]  Allan S. Cohen,et al.  Detection of Differential Item Functioning in the Graded Response Model , 1993 .

[5]  W. Meredith,et al.  Inferential Conditions in the Statistical Detection of Measurement Bias , 1992 .

[6]  Wen-Chung Wang,et al.  Factors Influencing the Mantel and Generalized Mantel-Haenszel Methods for the Assessment of Differential Item Functioning in Polytomous Items , 2004 .

[7]  G. Somes,et al.  The Generalized Mantel–Haenszel Statistic , 1986 .

[8]  Allan S. Cohen,et al.  Detection of Differential Item Functioning Under the Graded Response Model With the Likelihood Ratio Test , 1998 .

[9]  Hua-Hua Chang,et al.  Detecting DIF for Polytomously Scored Items: An Adaptation of the SIBTEST Procedure , 1995 .

[10]  Randall D. Penfield.  DIFAS: Differential Item Functioning Analysis System. Computer Program Exchange , 2005 .

[11]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[12]  Randall D. Penfield.  Assessing Differential Step Functioning in Polytomous Items Using a Common Odds Ratio Estimator. , 2007 .

[13]  Randall D. Penfield.  DIFAS: Differential Item Functioning Analysis System , 2005 .

[14]  Nambury S. Raju,et al.  A Description and Demonstration of the Polytomous-DFIT Framework , 1999 .

[15]  Roger E. Millsap,et al.  On the misuse of manifest variables in the detection of measurement bias , 1992 .

[16]  Neil J. Dorans,et al.  DIF Assessment for Polytomously Scored Items: A Framework for Classification and Evaluation , 1995 .

[17]  G. Masters.  The Partial Credit Model , 2016 .

[18]  D. Cox.  The Regression Analysis of Binary Sequences , 1958 .

[19]  James Algina,et al.  Applying the Liu-Agresti Estimator of the Cumulative Common Odds Ratio to DIF Detection in Polytomous Items , 2003 .

[20]  Gideon J. Mellenbergh,et al.  Conceptual Notes on Models for Discrete Polytomous Item Responses , 1995 .

[21]  Hua-Hua Chang,et al.  The unique correspondence of the item response function and item category response functions in polytomously scored item response models , 1994 .

[22]  Randall D. Penfield,et al.  Using a Taxonomy of Differential Step Functioning to Improve the Interpretation of DIF in Polytomous Items: An Illustration , 2008 .

[23]  A Agresti,et al.  Mantel-Haenszel-type inference for cumulative odds ratios with a stratified ordinal response. , 1996, Biometrics.

[24]  Steven P. Isham,et al.  A Comparison of Procedures to Detect Item Parameter Drift , 1998 .

[25]  Randall D. Penfield,et al.  An NCME Instructional Module on Using Differential Step Functioning to Refine the Analysis of DIF in Polytomous Items , 2009 .

[26]  Randall D. Penfield.  An Odds Ratio Approach for Assessing Differential Distractor Functioning Effects under the Nominal Response Model , 2008 .

[27]  Gregory Camilli,et al.  Application of a Method of Estimating DIF for Polytomous Test Items , 1999 .

[28]  Robert D. Ankenmann,et al.  An Investigation of the Power of the Likelihood Ratio Goodness-of-Fit Statistic in Detecting Differential Item Functioning. , 1999 .

[29]  Randall D. Penfield.  Three Classes of Nonparametric Differential Step Functioning Effect Estimators , 2008 .

[30]  F. Samejima.  Graded Response Model , 1997 .

[31]  Timothy R. Miller,et al.  Logistic Regression and Its Use in Detecting Differential Item Functioning in Polytomous Items , 1996 .

[32]  J. Spray,et al.  Logistic Discriminant Function Analysis for DIF Identification of Polytomously Scored Items , 1993 .

[33]  Christine E. DeMars.  Detection of Item Parameter Drift over Multiple Test Administrations , 2004 .

[34]  Nambury S. Raju,et al.  The area between two item characteristic curves , 1988 .

[35]  Rebecca Zwick,et al.  Evaluating the Magnitude of Differential Item Functioning in Polytomous Items , 1996 .

[36]  R. Zwick,et al.  Assessment of Differential Item Functioning for Performance Tasks , 1993 .

[37]  Daniel M. Bolt,et al.  A Monte Carlo Comparison of Parametric and Nonparametric Polytomous DIF Detection Methods , 2002 .

[38]  Allan S. Cohen,et al.  DIF Detection and Effect Size Measures for Polytomously Scored Items , 2007 .