Increasing Measurement Precision of PISA Through Multistage Adaptive Testing

Multistage adaptive testing (MST) designs are becoming more common in international large-scale assessments (ILSAs). This study presents what design considerations were made for introducing MST into the Programme for International Student Assessment (PISA) in the 2018 cycle. First, we conducted an experimental study during the field trial to examine the invariance of item parameters by unit order. The results confirmed that unit order effects were negligible, thus, confirming the feasibility of introducing the MST into the main survey. Second, we evaluated item parameter recovery and expected gains in measurement precision based on the PISA 2018 MST design through a simulation study. The simulation study revealed an acceptable level of parameter recovery and improved measurement precision, about 4–5% compared to the non-adaptive realistic benchmark in which students randomly took the same number of units.