The Subset Selection Technique for Multiple-Choice Tests: An Empirical Inquiry.

The impact of the Subset Selection Technique (SST) for administering and scoring multiple-choice items on certain properties of a test was compared with that of the two other commonly used methods, the Number Right (NR) and the Correction for Guessing Formula (CFG). Under SST, examinees are instructed to select any number of response alternatives, the objective being to include the correct answer in the chosen set. The effects of each scoring method on the psychometric properties of a test and on the performance of examinees with different achievement levels and/or risk-taking propensities were investigated. Results indicated that SST outperformed the other two methods, producing not only higher reliability and validity coefficients for the test, but doing so without favoring high risk takers. The superiority of SST may be attributed to two interrelated factors: the efficiency of the technique in controlling for guessing and the encouragement provided examinees to use their partial knowledge in responding.

[1]  Leverne S. Collet.  Elimination Scoring: An Empirical Evaluation. , 1971 .

[2]  Robert B. Frary,et al.  Elimination of the Guessing Component of Multiple-Choice Test Scores: Effect on Reliability and Validity , 1969 .

[3]  Leonard S. Feldt,et al.  A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample , 1980 .

[4]  John Schmid,et al.  Some Modifications of the Multiple-Choice Item , 1953 .

[5]  Ingram Olkin,et al.  A subset selection technique for scoring items on a multiple choice test , 1979 .

[6]  G. A. Ferguson,et al.  Statistical analysis in psychology and education , 1960 .

[7]  Frederic M. Lord,et al.  FORMULA SCORING AND NUMBER-RIGHT SCORING1 , 1975 .

[8]  R. Frary.  The Effect of Misinformation, Partial Information, and Guessing on Expected Multiple-Choice Test Item Scores , 1980 .

[9]  L. Bliss,et al.  A Test of Lord's Assumption Regarding Examinee Guessing Behavior on Multiple-Choice Tests Using Elementary School Students. , 1980 .

[10]  J. C. Arnold,et al.  On Scoring Multiple Choice Exams Allowing for Partial Knowledge , 1970 .

[11]  Clyde H. Coombs,et al.  On the Use of Objective Examinations , 1953 .

[12]  Clyde H. Coombs,et al.  The Assessment of Partial Knowledge1 , 1956 .

[13]  P. H. Jackson,et al.  Statistical analysis in psychology and education , 1960 .

[14]  M. Slakter.  Risk Taking on Objective Examinations1 , 1967 .

[15]  M. Slakter.  THE EFFECT OF GUESSING STRATEGY ON OBJECTIVE TEST SCORES1 , 1968 .

[16]  William H. Angoff,et al.  A Study of Hypotheses Basic to the Use of Rights and Formula Scores. Phase I--Based on Experimental Administration of College Board Tests [and] Phase II--Based on Operational Administration of the GMAT. , 1982 .

[17]  R. Frary,et al.  AN EMPIRICAL TEST OF LORD'S THEORETICAL RESULTS REGARDING FORMULA SCORING OF MULTIPLE‐CHOICE TESTS , 1977 .

[18]  Edward E. Cureton,et al.  The Correction for Guessing , 1966 .