An Investigation of the Impact of Misrouting under Two-Stage Multistage Testing: A Simulation Study. Research Report. ETS RR-14-01.

The purpose of this study was to investigate the potential impact of misrouting under a 2-stage multistage test (MST) design, which includes 1 routing and 3 second-stage modules. Simulations were used to create a situation in which a large group of examinees took each of the 3 possible MST paths (high, middle, and low). We compared differences in examinees' scores associated with different paths under 2 MST assembly conditions: a small-difference condition in which 3 modules at Stage 2 overlap in difficulty, and a large-difference condition in which the 3 modules at Stage 2 are distinct in difficulty. We also compared examinees' score differences associated with the target MST path with the ones obtained when the second-stage module was 1 level of difficulty above or below the target module. When the second-stage modules overlapped in difficulty, the score differences (i.e., bias) associated with different MST paths were negligible for practical purposes. Similar trends appeared even when the second-stage modules were significantly distinct in difficulty. The impact of misrouting was generally minimal under the MST design used in this study.