Item Characteristic Curve Asymmetry: A Better Way to Accommodate Slips and Guesses Than a Four-Parameter Model?

Four-parameter models have received increasing psychometric attention in recent years, as a reduced upper asymptote for item characteristic curves can be appealing for measurement applications such as adaptive testing and person-fit assessment. However, applications can be challenging due to the large number of parameters in the model. In this article, we demonstrate in the context of mathematics assessments how the slip and guess parameters of a four-parameter model may often be empirically related. This observation also has a psychological explanation to the extent that both asymptote parameters may be manifestations of a single item complexity characteristic. The relationship between lower and upper asymptotes motivates the consideration of an asymmetric item response theory model as a three-parameter alternative to the four-parameter model. Using actual response data from mathematics multiple-choice tests, we demonstrate the empirical superiority of a three-parameter asymmetric model in several standardized tests of mathematics. To the extent that a model of asymmetry ultimately portrays slips and guesses not as purely random but rather as proficiency-related phenomena, we argue that the asymmetric approach may also have greater psychological plausibility.

[1]  H. Bolfarine,et al.  A skew item response model , 2006 .

[2]  Paul De Boeck,et al.  IRT Models for Ability-Based Guessing , 2006 .

[3]  Steven Andrew Culpepper,et al.  A Restricted Four-Parameter IRT Model: The Dyad Four-Parameter Normal Ogive (Dyad-4PNO) Model , 2020, Psychometrika.

[4]  J. D. L. Torre,et al.  The Generalized DINA Model Framework. , 2011 .

[5]  F. Lord Applications of Item Response Theory To Practical Testing Problems , 1980 .

[6]  Steven Andrew Culpepper The Prevalence and Implications of Slipping on Low-Stakes, Large-Scale Assessments , 2017 .

[7]  S. Culpepper Revisiting the 4-Parameter Item Response Model: Bayesian Estimation and Application , 2016, Psychometrika.

[8]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[9]  Fumiko Samejima,et al.  Logistic positive exponent family of models: Virtue of asymmetric item characteristic curves , 2000 .

[10]  Gongjun Xu,et al.  Marginalized maximum a posteriori estimation for the four-parameter logistic model under a mixture modelling framework. , 2020, The British journal of mathematical and statistical psychology.

[11]  Frederic M. Lord,et al.  An Upper Asymptote for the Three-Parameter Logistic Item-Response Model. , 1981 .

[12]  S. Reise,et al.  How many IRT parameters does it take to model psychopathology items? , 2003, Psychological methods.

[13]  D. Molenaar Heteroscedastic Latent Trait Models for Dichotomous Data , 2015, Psychometrika.

[14]  Niels G. Waller,et al.  Measuring psychopathology with non-standard IRT models: Fitting the four-parameter model to the MMPI , 2010 .

[15]  D. Bolt,et al.  An Alternative to the 3PL: Using Asymmetric Item Characteristic Curves to Address Guessing Effects. , 2018 .

[16]  T. P. Hutchinson Ability, partial information, guessing : statistical modelling applied to multiple, choice tests , 1991 .

[17]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[18]  Yung-Chin Yen,et al.  An Empirical Evaluation of the Slip Correction in the Four Parameter Logistic Models With Computerized Adaptive Testing , 2012 .

[19]  Eric Loken,et al.  Estimation of a four-parameter item response theory model. , 2010, The British journal of mathematical and statistical psychology.

[20]  D. Bolt,et al.  Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses , 2018, Psychometrika.

[21]  Jonathan Templin,et al.  Diagnostic Measurement: Theory, Methods, and Applications , 2010 .

[22]  D. Bolt,et al.  IRT Model Misspecification and Measurement of Growth in Vertical Scaling. , 2014 .