Detection of Differential Item Functioning Using the Lasso Approach

This article proposes a novel approach to detect differential item functioning (DIF) among dichotomously scored items. Unlike standard DIF methods that perform an item-by-item analysis, we propose the “LR lasso DIF method”: logistic regression (LR) model is formulated for all item responses. The model contains item-specific intercepts, an effect of the sum score, and item-group interaction (i.e., DIF) effects, with a lasso penalty on all DIF parameters. Optimal penalty parameter selection is investigated through several known information criteria (Akaike information criterion, Bayesian information criterion, and cross validation) as well as through a newly developed alternative. A simulation study was conducted to compare the global performance of the suggested LR lasso DIF method to the LR and Mantel–Haenszel methods (in terms of false alarm and hit rates). It is concluded that for small samples, the LR lasso DIF approach globally outperforms the LR method, and also the Mantel–Haenszel method, especially in the presence of item impact, while it yields similar results with larger samples.

[1]  Gerhard Tutz,et al.  A Penalty Approach to Differential Item Functioning in Rasch Models , 2015, Psychometrika.

[2]  Neil J. Dorans,et al.  Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. , 1986 .

[3]  David Magis,et al.  Angoff's delta method revisited: improving DIF detection under small samples. , 2012, The British journal of mathematical and statistical psychology.

[4]  A. Rupp,et al.  Impact of Missing Data on the Detection of Differential Item Functioning , 2009 .

[5]  P. Boeck,et al.  A general framework and an R package for the detection of dichotomous differential item functioning , 2010, Behavior research methods.

[6]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[7]  H. Swaminathan,et al.  Detecting Differential Item Functioning Using Logistic Regression Procedures , 1990 .

[8]  Tiee-Jian Wu,et al.  The weighted average information criterion for multivariate regression model selection , 2013, Signal Process..

[9]  H. Zou The Adaptive Lasso and Its Oracle Properties , 2006 .

[10]  H. Bozdogan Choosing the Number of Component Clusters in the Mixture-Model Using a New Informational Complexity Criterion of the Inverse-Fisher Information Matrix , 1993 .

[11]  David Magis,et al.  RIM: A random item mixture model to detect Differential Item Functioning , 2010 .

[12]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[13]  D. Weakliem A Critique of the Bayesian Information Criterion for Model Selection , 1999 .

[14]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[15]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[16]  P. De Boeck,et al.  Identification of Differential Item Functioning in Multiple-Group Settings: A Multivariate Outlier Detection Approach , 2011, Multivariate behavioral research.

[17]  J. López-Pina,et al.  Differential Item Functioning Detection and Effect Size: A Comparison between Logistic Regression and Mantel-Haenszel Procedures , 2004 .

[18]  Christine E. DeMars Type I Error Inflation for Detecting DIF in the Presence of Impact , 2010 .

[19]  Randall D. Penfield Assessing Differential Item Functioning Among Multiple Groups: A Comparison of Three Mantel-Haenszel Procedures , 2001 .

[20]  Effect of Multiple Testing Adjustment in Differential Item Functioning Detection , 2013 .

[21]  H. Bozdogan Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[22]  Gregory L. Candell,et al.  An Iterative Procedure for Linking Metrics and Assessing Item Bias in Item Response Theory , 1988 .

[23]  I. J. Myung,et al.  When a good fit can be bad , 2002, Trends in Cognitive Sciences.

[24]  Brian F. French,et al.  Detection of Crossing Differential Item Functioning , 2007 .

[25]  Nambury S. Raju,et al.  Determining the Significance of Estimated Signed and Unsigned Areas Between Two Item Response Functions , 1990 .

[26]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[27]  Dorothy T. Thayer,et al.  Differential Item Performance and the Mantel-Haenszel Procedure. , 1986 .

[28]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[29]  David Magis,et al.  A Robust Outlier Approach to Prevent Type I Error Inflation in Differential Item Functioning , 2012 .

[30]  Nambury S. Raju,et al.  The area between two item characteristic curves , 1988 .

[31]  Wen-Chung Wang,et al.  Effects of Average Signed Area Between Two Item Characteristic Curves and Test Purification Procedures on the DIF Detection via the Mantel-Haenszel Method , 2004 .

[32]  David Thissen,et al.  Quick and Easy Implementation of the Benjamini-Hochberg Procedure for Controlling the False Positive Rate in Multiple Comparisons , 2002 .

[33]  Paul De Boeck,et al.  On the Relationships between Sum Score Based Estimation and Joint Maximum Likelihood Estimation , 2008 .

[34]  R. Hambleton,et al.  The Effects of Purification of Matching Criterion on the Identification of DIF Using the Mantel-Haenszel Procedure , 1993 .

[35]  Tiee-Jian Wu,et al.  The weighted average information criterion for order selection in time series and regression models , 1998 .

[36]  William Stout,et al.  A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF , 1993 .

[37]  N. Perkins,et al.  The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve. , 2006, American journal of epidemiology.

[38]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[39]  Holmes W. Finch The Use of Multiple Imputation for Missing Data in Uniform DIF Analysis: Power and Type I Error Rates , 2011 .

[40]  P. De Boeck,et al.  Type I Error Inflation in DIF Identification With Mantel–Haenszel , 2014 .

[41]  F. Lord Applications of Item Response Theory To Practical Testing Problems , 1980 .

[42]  Howard Wainer,et al.  Use of item response theory in the study of group differences in trace lines. , 1988 .

[43]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[44]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[45]  H. Swaminathan,et al.  A Comparison of Logistic Regression and Mantel-Haenszel Procedures for Detecting Differential Item Functioning , 1993 .

[46]  David Magis,et al.  A Generalized Logistic Regression Procedure to Detect Differential Item Functioning Among Multiple Groups , 2011 .