A Strategy for Replacing Sum Scoring

This article promotes the use of modern test theory in testing situations where sum scores for binary responses are now used. It directly compares the efficiencies and biases of classical and modern test analyses and finds an improvement in the root mean squared error of ability estimates of about 5% for two designed multiple-choice tests and about 12% for a classroom test. A new parametric density function for ability estimates, the tilted scaled β, is used to resolve the nonidentifiability of the univariate test theory model. Item characteristic curves (ICCs) are represented as basis function expansions of their log-odds transforms. A parameter cascading method along with roughness penalties is used to estimate the corresponding log odds of the ICCs and is demonstrated to be sufficiently computationally efficient that it can support the analysis of large data sets.

[1]  J. Ramsay.  Kernel smoothing approaches to nonparametric item characteristic curve estimation , 1991 .

[2]  James O. Ramsay,et al.  A Functional Approach to Modeling Test Data , 1997 .

[3]  Young-Sun Lee,et al.  A Comparison of Methods for Nonparametric Estimation of Item Characteristic Curves for Binary Items , 2007 .

[4]  James O. Ramsay,et al.  Nonparametric Item Response Function Estimates with the EM Algorithm , 2002 .

[5]  David Thissen,et al.  Item Response Theory for Items Scored in Two Categories , 2001 .

[6]  Carol M Woods,et al.  Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables. , 2006, Psychological methods.

[7]  Jiguo Cao,et al.  Parameter estimation for differential equations: a generalized smoothing approach , 2007 .

[8]  L. L. Thurstone,et al.  A method of scaling psychological and educational tests. , 1925 .

[9]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[10]  .. W. V. Der,et al.  On Profile Likelihood , 2000 .

[11]  Hans-Georg Müller,et al.  Functional Data Analysis , 2016 .

[12]  W. Wong,et al.  Profile Likelihood and Conditionally Parametric Models , 1992 .

[13]  H. Gulliksen.  Theory of mental tests , 1952 .

[14]  James O. Ramsay,et al.  A GEOMETRICAL APPROACH TO ITEM RESPONSE THEORY , 1996 .

[15]  Douglas M. Bates,et al.  Nonlinear Regression Analysis and Its Applications , 1988 .

[16]  Marie Frei,et al.  Functional Data Analysis With R And Matlab , 2016 .

[17]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[18]  David Thissen,et al.  Item Response Theory with Estimation of the Latent Population Distribution Using Spline-Based Densities , 2006, Psychometrika.

[19]  Robert J. Mokken,et al.  Nonparametric Models for Dichotomous Responses , 1997 .