Bayesian Ability Estimation via 3PL (Three-Parameter Logistic) with Partially Known Item Parameters

Abstract : A standard practice in mental testing is to score individuals using the responses to a set of test items which have previously been calibrated. When latent trait models are employed, the calibration involves estimating parameters of the model using a moderately large sample. The estimated parameters are then assumed to be the true values when the scoring is performed. Even when the assumed model is correct, there are two sources of errors in this process. One is due to the responses of the individuals being scored and the other is due to the error in the calibration. Ignoring the second source could lead to inferential errors, particularly when the calibrating sample is not large. In many areas of testing large samples may not be readily available for calibration. Moreover, disclosure laws commonly require public dissemination of tests, making it necessary to have more items while the pool from which to draw the calibrating sample is limited. This paper deals with the problem of estimating ability when there is uncertainty concerning the item parameters due to the limited size of the calibrating sample. Because of the sequential nature of first calibrating the test and then using it on the target population, the Bayesian paradigm for statistical inference is particularly attractive. This paper discusses how the uncertainty in the item parameters may be incorporated into the estimation and uncertainty of the abilities being measured.