A Bayesian Method for the Detection of Item Preknowledge in CAT. Law School Admission Council Computerized Testing Report. LSAC Research Report Series.

With the increased use of computerized adaptive testing, which allows for continuous testing, new concerns about test security have evolved, one being the assurance that items in an item pool are safeguarded from theft. In this paper, the risk of score inflation and procedures to detect test takers using item preknowledge are explored. When test takers use preknowledge of items, their item responses may deviate from the underlying item response theory (IRT) model, and estimated abilities may be inflated. This deviation may be detected through the use of person-fit indices. A Bayesian posterior log odd ratio index is proposed for detecting the use of item preknowledge. In this approach to person-fit, the estimated probability that each test taker has preknowledge of items is updated after each item response. These probabilities are based on IRT parameters, a model specifying the probability that each item has been memorized, and the test taker's item responses. Simulations based on an operational computerized adaptive test pool were used to demonstrate the risk of item preknowledge to test security and the use of the odds ratio index. An appendix discusses the three classes of models used in the study. (Author/SLD) Reproductions supplied by EDRS are the best that can be made from the original document.