The Effect of Restricting Ability Distributions in the Estimation of Item Difficulties: Implications for a CAT Implementation.

Responses to previously calibrated items administered in a computerized adaptive testing (CAT) mode may be used to recalibrate the items. This live-data simulation study investigated the possibility, and limitations, of on-line adaptive recalibration of precalibrated items. Responses to items of a Rasch-based paper-and-pencil licensure examination were used to simulate CAT and paper-and-pencil administrations, defining CAT forms of varying difficulty levels and samples of various sizes. Forms were calibrated and new b-values were compared with the bank b-values obtained from responses of the reference group taking the paper-and-pencil examination. Results indicate that bank b-values were not well replicated when difficult items were calibrated using responses from able examinees and easy items were calibrated using responses from less able examinees. On-line adaptive recalibration as simulated in this study has limitations. However, a "modified" on-line adaptive recalibration may still be a possibility as long as: (1) a reasonably large CAT recalibration sample is predefined from the reference group; (2) the sample has a mean ability similar tu that of the reference group; and (3) items to be recalibrated together are relatively heterogeneous in Rasch difficulty. Eight tables are included. (Contains 3 references.) (Author/SLD) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. *********************************************************************** U.& DEPARTMENT OF EDUCATION Oft* of Educational Research and Improvement ECIUC RONAL P.'OURCES INFORMATION CENTER (ERIC) his document Is been rProduced as received from the parson or organization oflgalmvigo C Minor changes have been made to improve reproduction guild), Points ot view or opinions Mated in this docuC1 ment do not necessarily repubsent official OERI position or policy 12:1 -PERMISSION TO REPRODUCE THIS MATERIAL HAS BEEN GRANTED BY