Modelling response error in school effectiveness research

Statistical modelling of school effectiveness in educational research is considered. Variance component models are generally accepted for the analysis of such studies. A shortcoming is that outcome variables are still treated as measured without an error. Unreliable variables produce biases in the estimates of the other model parameters. The variability of the relationships across schools and the effects of schools on students’ outcomes differ substantially when taking the measurement error in the dependent variables of the variance component models into account. The random effects model can be extended to handle measurement error using a response model, leading to a random effects item response theory model. This extended random effects model is in particular suitable when subjects are measured repeatedly on the same outcome at several points in time.

[1]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[2]  H. Goldstein.  Multilevel Statistical Models , 2006 .

[3]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[4]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[5]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[6]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[7]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .

[8]  N. Longford.  A FAST SCORING ALGORITHM FOR MAXIMUM LIKELIHOOD ESTIMATION IN UNBALANCED MIXED MODELS WITH NESTED RANDOM EFFECTS , 1987 .

[9]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[10]  M. Aitkin,et al.  Statistical Modelling Issues in School Effectiveness Studies , 1986 .

[11]  R. Paap.  What are the advantages of MCMC based inference in latent variable models? , 2002 .

[12]  W. Gilks,et al.  Random-effects models, for longitudinal data using Gibbs sampling. , 1993, Biometrics.

[13]  J. Albert.  Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling , 1992 .

[14]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[15]  Gerardus Johannes Andre Fox.  Multilevel IRT: a Bayesian perspective on estimating parameters and testing statistical hypotheses , 2001 .

[16]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[17]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[18]  Brian W. Junker,et al.  Applications and Extensions of MCMC in IRT: Multiple Item Types, Missing Data, and Rated Responses , 1999 .

[19]  S. Doolaard.  Stability and Change in Results of Schooling , 2002 .

[20]  J. Fox,et al.  Bayesian estimation of a multilevel IRT model using gibbs sampling , 2001 .

[21]  Peter Congdon.  Bayesian statistical modelling , 2002 .

[22]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[23]  Estimation of true score and error variance for tests under various equivalence assumptions , 1969 .

[24]  H. Goldstein.  Multilevel mixed linear model analysis using iterative generalized least squares , 1986 .

[25]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[26]  Harvey Goldstein,et al.  League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance , 1996 .

[27]  F. Baker,et al.  Item response theory : parameter estimation techniques , 1993 .

[28]  Jean-Paul Fox,et al.  Bayesian modeling of measurement error in predictor variables using item response theory , 2003 .

[29]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[30]  W. Kristof.  Estimation of reliability and true score variance from a split of a test into three arbitrary parts , 1974 .

[31]  P. Jackson.  The estimation of true score variance and error variance in the classical test theory model , 1973 .

[32]  R. Jennrich,et al.  Unbalanced repeated-measures models with structured covariance matrices. , 1986, Biometrics.

[33]  I. W. Molenaar,et al.  Data, model, conclusion, doing it again , 1998 .

[34]  A. Béguin,et al.  MCMC estimation of multidimensional IRT models , 1998 .

[35]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[37]  A. Béguin,et al.  MCMC estimation and some model-fit analysis of multidimensional IRT models , 2001 .