Adjusting for information inflation due to local dependency in moderately large item clusters

When multiple items are clustered around a reading passage, the local independence assumption in item response theory is often violated. The amount of information contained in an item cluster is usually overestimated if violation of local independence is ignored and items are treated as locally independent when in fact they are not. In this article we provide a general method that adjusts for the inflation of information associated with a test containing item clusters. A computational scheme was presented for the evaluation of the factor of adjustment for clusters in the restrictive case of two items per cluster, and the general case of more than two items per cluster. The methodology was motivated by a study of the NAEP Reading Assessment. We present a simulated study along with an analysis of a NAEP data set.

[1]  Wendy M. Yen,et al.  Effects of Local Item Dependence on the Fit and Equating Performance of the Three-Parameter Logistic Model , 1984 .

[2]  Ross L. Prentice,et al.  Binary Regression Using an Extended Beta-Binomial Distribution, with Discussion of Correlation Induced by Covariate Measurement Errors , 1986 .

[3]  R. Prentice,et al.  Correlated binary regression with covariates specific to each binary observation. , 1988, Biometrics.

[4]  P. Holland,et al.  Discrete Multivariate Analysis. , 1976 .

[5]  J. Chambers,et al.  The New S Language , 1989 .

[6]  S. Zeger,et al.  Multivariate Regression Analyses for Categorical Data , 1992 .

[7]  Stephen G. Sireci,et al.  ON THE RELIABILITY OF TESTLET‐BASED TESTS , 1991 .

[8]  Wendy M. Yen,et al.  Scaling Performance Assessments: Strategies for Managing Local Item Dependence , 1993 .

[9]  Howard Wainer,et al.  Precision and Differential Item Functioning on a Testlet-Based Test: The 1991 Law School Admissions Test as an Example , 1995 .

[10]  William Stout,et al.  A New Item Response Theory Modeling Approach with Applications to Unidimensionality Assessment and Ability Estimation , 1990 .

[11]  B. Junker.  Conditional association, essential independence and monotone unidimensional Item response models , 1993 .

[12]  William H. Press,et al.  Numerical recipes in C , 2002 .

[13]  Eric T. Bradlow.  Teacher’s Corner: Negative Information and The Three-Parameter Logistic Model , 1996 .

[14]  R. Clarke,et al.  Theory and Applications of Correspondence Analysis , 1985 .

[15]  Richard A. Becker,et al.  The New S Language , 1989 .

[16]  L. Fahrmeir,et al.  Multivariate statistical modelling based on generalized linear models , 1994 .

[17]  田中 豊.  Multivariate Statistical Modelling Based on Generalized Linear Models/Ludwig Fahrmeir,Gerhard Tutz(1994) , 1995 .

[18]  Hongsheng Wu.  Some issues in item response theory , 1996 .

[19]  Eric T. Bradlow.  Negative Information and the Three-Parameter Logistic Model. Teacher's Corner. , 1996 .

[20]  William Stout,et al.  A nonparametric approach for assessing latent trait unidimensionality , 1987 .

[21]  Patrick Suppes,et al.  When are Probabilistic Explanations Possible , 1981 .

[22]  P. Rosenbaum,et al.  Conditional Association and Unidimensionality in Monotone Latent Variable Models , 1985 .

[23]  L. Zhao,et al.  Correlated binary regression using a quadratic exponential model , 1990 .

[24]  Andrea Rotnitzky,et al.  Regression Models for Discrete Longitudinal Responses , 1993 .

[25]  D. Cox.  The Analysis of Multivariate Binary Data , 1972 .

[26]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[27]  J. Dale.  Global cross-ratio models for bivariate, discrete, ordered responses. , 1986, Biometrics.

[28]  Brian W. Junker,et al.  Essential independence and likelihood-based ability estimation for polytomous items , 1991 .

[29]  Stephen E. Fienberg,et al.  Discrete Multivariate Analysis: Theory and Practice , 1976 .

[30]  Howard Wainer,et al.  Item Clusters and Computerized Adaptive Testing: A Case for Testlets , 1987 .

[31]  Rebecca Zwick,et al.  Assessing the Dimensionality of NAEP Reading Data , 1987 .

[32]  G. L. Ritter,et al.  Density Estimations and the Characterization of Binary Infrared Spectra , 1975 .

[33]  P. Diggle,et al.  Analysis of Longitudinal Data , 2003 .

[34]  B. Efron,et al.  Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information , 1978 .

[35]  Brian Habing,et al.  Conditional Covariance-Based Nonparametric Multidimensionality Assessment , 1996 .