Using Explanatory Item Response Theory Modeling to Investigate Context Effects of Differential Item Functioning for Students With Disabilities

To address whether or not modifications in test administration influence item functioning for students with disabilities on a high-stakes statewide problem-solving assessment, a sample of 868 students (with and without disabilities) from 74 Georgia schools were randomly assigned to one of three testing conditions (resource guide, calculator, or standard administration). The authors describe a conceptual framework for investigating measurement invariance, specifically differential item functioning within the context of assessing students with disabilities. Specifically, we illustrate how the hierarchical generalized linear model, an explanatory model that incorporates item response models into hierarchical models in multilevel settings, can be used to explore issues of measurement invariance traditionally addressed using descriptive item response models, that is, the many-facet Rasch model. Results obtained from the two approaches are reported and discussed.

[1]  J. Bielinski,et al.  Technical Report 31 Read-Aloud Accommodation : Effects on Multiple-Choice Reading and Math Items , 2012 .

[2]  G. Masters,et al.  Rating Scale Analysis. Rasch Measurement. , 1983 .

[3]  Gail F. Munger,et al.  Effect of Speededness on Test Performance of Handicapped and Nonhandicapped Examinees , 1991 .

[4]  Robert J. Sampson,et al.  6. A Multivariate, Multilevel Rasch Model with Application to Self-Reported Criminal Behavior , 2003 .

[5]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data: Little/Statistical Analysis with Missing Data , 2002 .

[6]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data , 1988 .

[7]  Y. F. Cheong.  Analysis of School Context Effects on Differential Item Functioning Using Hierarchical Generalized Linear Models , 2006 .

[8]  Randy Elliot Bennett,et al.  Differential Item Functioning on the SAT‐M Braille Edition , 1989 .

[9]  H. Huynh,et al.  Performance of Students With Disabilities Under Regular and Oral Administrations of a High-Stakes Reading Examination , 2006 .

[10]  G. Engelhard,et al.  Performance of Students With and Without Disabilities Under Modified Conditions , 2010 .

[11]  George Engelhard,et al.  Using Item Response Theory and Model—Data Fit to Conceptualize Differential Item and Person Functioning for Students With Disabilities , 2009 .

[12]  G. Engelhard,et al.  Using Confirmatory Factor Analysis and the Rasch Model to Assess Measurement Invariance in a High Stakes Reading Assessment , 2010 .

[13]  David A. Frisbie,et al.  The Validity of ITBS Reading Comprehension Test Scores for Learning Disabled and Non Learning Disabled Students under Extended-Time Conditions. , 2000 .

[14]  A. Cohen,et al.  The Role of Extended Time and Item Content on a High–Stakes Mathematics Test , 2005 .

[15]  Yuk Fai Cheong,et al.  HLM 6: Hierarchical Linear and Nonlinear Modeling , 2000 .

[16]  Russell V. Lenth,et al.  Statistical Analysis With Missing Data (2nd ed.) (Book) , 2004 .

[17]  Stephen W. Raudenbush,et al.  1. Ecometrics: Toward a Science of Assessing Ecological Settings, with Application to the Systematic Social Observation of Neighborhoods , 1999 .

[18]  B. Elbaum,et al.  Effects of an Oral Testing Accommodation on the Mathematics Performance of Secondary Students With and Without Learning Disabilities , 2007 .

[19]  M. Thurlow,et al.  Read-Aloud Accommodation: Effects on Multiple-Choice Reading and Math Items. Technical Report. , 2001 .

[20]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[21]  S. Raudenbush,et al.  Measurement and structural models for children's problem behaviors. , 2000, Psychological methods.

[22]  Akihito Kamata,et al.  Item Analysis by the Hierarchical Generalized Linear Model. , 2001 .

[23]  R. Linn,et al.  INTERACTIONS BETWEEN ITEM CONTENT AND GROUP MEMBERSHIP ON ACHIEVEMENT TEST ITEMS , 1981 .

[24]  J. Linacre,et al.  Many-facet Rasch measurement , 1994 .

[25]  M K Runyan,et al.  The Effect of Extra Time on Reading Comprehension Scores for University Students With and Without Learning Disabilities , 1991, Journal of learning disabilities.

[26]  Dorinda J. Gallant,et al.  Comparability of Student Performance Between Regular and Oral Administrations for a High-Stakes Mathematics Test , 2004 .

[27]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[28]  S. Bolt,et al.  Comparing DIF Across Math and Reading/Language Arts Tests for Students Receiving a Read-Aloud Accommodation , 2006 .

[29]  Maria Elena Arguelles,et al.  Effects of a Student-Reads-Aloud Accommodation on the Performance of Students With and Without Learning Disabilities on a Test of Reading Comprehension , 2004, Large-scale Testing of Students With Disabilities.

[30]  R. Bennett,et al.  THE INTERNAL CONSTRUCT VALIDITY OF THE SAT ACROSS HANDICAPPED AND NONHANDICAPPED POPULATIONS , 1985 .