AN INVESTIGATION OF IRT-BASED ASSEMBLY OF THE TOEFL TEST

The current study, carried out jointly by Test Development and Statistical Analysis staff at Educational Testing Service (ETS), investigated the feasibility of the Automated Item Selection (AIS) procedure for the Test of English as a Foreign Language (TOEFL). IRT-based statistical specifications were developed. Two TOEFL® test forms were assembled using AIS and statistical and content-related properties were evaluated. The results of this study show that the statistical consistency (parallelism) of the tests assembled using AIS would appear to be superior to the consistency of tests assembled using traditional test-assembly procedures. The results of the study also provided strong evidence that AIS-assembled TOEFL tests can successfully meet the IRT-based specifications. Test Development staff observed visible gains in efficiency in item selection for Section 1 and Section 2 and the potential for time gains in Section 3. The implications of AIS for pool management were explored.