MULTISIB: A Procedure to Investigate DIF When a Test is Intentionally Two-Dimensional

MULTISIB is proposed as a statistical test for assessing differential item functioning (DIF) of intentionally two-dimensional test data, such as a mathematics test designed to measure algebra and geometry. MULTISIB is based on the multidimensional model of DIF as presented in Shealy & Stout (1993), and is a direct extension of SIBTEST, its unidimensional counterpart. For an intentionally two-dimensional test, DIF is appropriately modeled to result from secondary dimensional influence from other than the two intended dimensions. Simulation studies were used to assess the performance of MULTISIB to detect DIF in intentionally two-dimensional tests. These results indicate that MULTISIB exhibited reasonably good adherence to the nominal level of significance and good power. Moreover, for each DIF model the average amount of DIF estimated over the 100 simulations of the model by MULTISIB was close to the true value, confirming its relative lack of statistical estimation bias in assessing true DIF. In addition, the simulation studies supported the importance of using the regression correction to adjust the scores on the studied item due to impact and the importance of matching examinees on two subtest scores instead of the total test score.

[1]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[2]  W. Stout,et al.  A new procedure for detection of crossing DIF , 1996 .

[3]  Wim J. van der Linden,et al.  IRT-Based Internal Measures of Differential Functioning of Items and Tests , 1995 .

[4]  H. Swaminathan,et al.  Identification of Items that Show Nonuniform DIF , 1996 .

[5]  Hariharan Swaminathan,et al.  Performance of the Mantel-Haenszel and Simultaneous Item Bias Procedures for Detecting Differential Item Functioning , 1993 .

[6]  H. Swaminathan,et al.  A Comparison of Logistic Regression and Mantel-Haenszel Procedures for Detecting Differential Item Functioning , 1993 .

[7]  Ratna Nandakumar,et al.  Simultaneous DIF Amplification and Cancellation: Shealy-Stout's Test for DIF , 1993 .

[8]  H. Swaminathan,et al.  Detecting Differential Item Functioning Using Logistic Regression Procedures , 1990 .

[9]  William Stout,et al.  A Multidimensionality-Based DIF Analysis Paradigm , 1996 .

[10]  Colin Fraser,et al.  NOHARM: Least Squares Item Factor Analysis. , 1988, Multivariate behavioral research.

[11]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[12]  Terry A. Ackerman.  A Didactic Explanation of Item Bias, Item Impact, and Item Validity from a Multidimensional Perspective , 1992 .

[13]  William Stout,et al.  Simulation Studies of the Effects of Small Sample Size and Studied Item Parameters on SIBTEST and Mantel‐Haenszel Type I Error Performance , 1996 .

[14]  Terry A. Ackerman,et al.  A Didactic Example of the Influence of Conditioning on the Complete Latent Ability Space When Performing DIF Analyses. , 1993 .

[15]  Neil J. Dorans,et al.  Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. , 1986 .

[16]  Louis V. DiBello,et al.  A Kernel-Smoothed Version of SIBTEST With Applications to Local DIF Inference and Function Estimation , 1996 .

[17]  Jeffrey A Douglas,et al.  Item-Bundle DIF Hypothesis Testing: Identifying Suspect Bundles and Assessing Their Differential Functioning , 1996 .

[18]  Hua-Hua Chang,et al.  Detecting DIF for Polytomously Scored Items: An Adaptation of the SIBTEST Procedure , 1995 .

[19]  H. Wainer,et al.  Differential Item Functioning. , 1994 .

[20]  Terry Ackerman,et al.  Graphical Representation of Multidimensional Item Response Theory Analyses , 1996 .

[21]  The Definition of Difficulty and Discrimination for Multidimensional Item Response Theory Models. , 1983 .

[22]  David M. Williams,et al.  VALIDITY OF APPROXIMATION TECHNIQUES FOR DETECTING ITEM BIAS , 1985 .

[23]  Brian E. Clauser,et al.  Using logistic regression and the Mantel-Haenszel with multiple ability estimates to detect differential item functioning. , 1995 .

[24]  John Hattie,et al.  Methodology Review: Assessing Unidimensionality of Tests and ltenls , 1985 .

[25]  William Stout,et al.  A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF , 1993 .