Capitalization on chance in variable-length classification tests employing the Sequential Probability Ratio Test

The sequential probability ratio test (SPRT) is a popular termination criterion for variable-length classification tests. The SPRT is often paired with cut-based item selection in which item information is maximized at the cut point. However, items are chosen on the basis of their parameter estimates, and capitalization on chance may occur. We investigated the effects of capitalization on chance on test length and classification accuracy in several variable-length test simulations. In addition to capitalizing on large discrimination estimates, the item selection criterion chose items with difficulty estimates systematically higher or lower than their true difficulty values. This capitalization on chance had non-negligible effects on both test length and classification accuracy and induced an inverse relationship between them, though the particular effects were highly sensitive to the cut location. The results also indicate that implementing item exposure control effectively reduced the effects of capitalization on chance on testing outcomes.