Producing equivalent examination forms : an assessment of the British Columbia Ministry of Education examination construction procedure

Questions have been raised concerning the equivalency of the January, June, and August forms of the British Columbia provincial Grade 12 examinations for a given subject. The procedure for constructing these examinations has been changed as of the 1990/91 school year. The purpose of this study was to duplicate this new procedure and assess the equivalency of the forms that resulted. An examination construction team, all of whom had previous experience with the British Columbia Ministry of Education's Student Assessment Branch, simultaneously constructed two forms of a Biology 12 examination from a common table of specifications using a pool of multiple choice items from previous examinations. A sample of students was obtained in the Okanagan, Thompson, and North Thompson areas of British Columbia. Both forms were administered to each student, as required by the test equating design (Design II (Angoff, 1971)) chosen. The data sample consisted of responses from 286 students. The data were analyzed using a classical item analysis (LERTAP, Nelson, 1974) followed by a 2x2 order-by-form fixed effects ANOVA with repeated measures on the second factor. Item analysis revealed all items on both forms performed satisfactorily, ruling out an alternate hypothesis of flawed items being the cause of the lack of equivalence found. Results showed a significant (p<.05) difference in the means of the two forms, no

[1]  UNINTERPRETABLE SCORES: THEIR IMPLICATIONS FOR TESTING PRACTICE , 1981 .

[2]  C. Hoyt.  Test reliability estimated by analysis of variance , 1941 .

[3]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[4]  H. Gulliksen.  Theory of mental tests , 1952 .

[5]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[6]  F. Lord.  NOTES ON COMPARABLE SCALES FOR TEST SCORES , 1950 .

[7]  John C. Bianchini,et al.  Anchor Test Study. Equivalence and Norms Tables for Selected Reading Achievement Tests (Grades 4, 5, 6). , 1974 .

[8]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[9]  Standard Errors of Equipercentile Equating for the Common Item Nonequivalent Populations Design , 1985 .

[10]  D. Whitney,et al.  Comparison of Four Procedures for Equating the Tests of General Educational Development. , 1982 .

[11]  Susy Macqueen,et al.  Validity , 1973, Just Algorithms.

[12]  Checking the Equivalence of Nearly Identical Test Editions. , 1988 .

[13]  R. Hambleton.  Principles and selected applications of item response theory. , 1989 .

[14]  John O. Anderson.  The Impact of Provincial Examinations on Education in British Columbia: General Report. , 1990 .

[15]  G. Glass,et al.  Statistical methods in education and psychology , 1970 .

[16]  F. Lord.  The Standard Error of Equipercentile Equating , 1981 .

[17]  B. Bloom.  Taxonomy of educational objectives , 1956 .

[18]  Barbara J. Holmes.  Individually-administered intelligence tests : an application of anchor test norming and equating procedures in British Columbia , 1981 .

[19]  M. Albanese.  The Projected Impact of the Correction for Guessing on Individual Scores. , 1988 .