A Nominal Response Model Approach for Detecting Answer Copying

When examinees copy answers to test questions from other examinees, the validity of the test is compromised. Most available statistical procedures for detecting copying were developed out of classical test theory (CrT); hence, they suffer from sampledependent score and item statistics, and biased estimates of the expected number of answer matches between a pair of examinees. Item response theory (IRT) based procedures alleviate these problems; however, because they fail to compare the similarity of responses between neighboring examinees, they have relatively poor power for detecting copiers. A new IRT-based test statistic, wo, was compared with the best CUT-based index g2 under various copying conditions, amounts of copying, test lengths, and sample sizes. w consistently held the Type I error rate at or below the nominal level; g2 yielded substantially inflated Type I error rates. The power of w varied as a function of both test length and the percentage of items copied. w demonstrated good power to detect copiers, provided that at least 20% of the items were copied on an 80-item test and at least 30% were copied on a 40-item test. Based on these results, with regard to both Tbype I error rate and power, c appears to be more useful than g2 as a copying index.

[1]  T. Nicolaus Tideman,et al.  Indices of Cheating on Multiple-Choice Tests , 1977 .

[2]  Karen S. Nantz,et al.  Social Accounts and Metaphors about Cheating , 1994 .

[3]  Fritz Drasgow,et al.  Appropriateness measurement with polychotomous item response models and standardized indices , 1984 .

[4]  Robert L. Brennan,et al.  A Comparison of Several Statistical Methods for Examining Allegations of Copying , 1987 .

[5]  J. S. Baird,et al.  Current trends in college cheating , 1980 .

[6]  Joel R. Levin,et al.  New developments in pairwise multiple comparisons : some powerful and practicable procedures , 1991 .

[7]  William H. Angoff,et al.  The Development of Statistical Indices for Detecting Cheaters , 1974 .

[8]  Donald B. Rubin,et al.  Measuring the Appropriateness of Multiple-Choice Test Scores , 1979 .

[9]  F. Baker,et al.  Item response theory : parameter estimation techniques , 1993 .

[10]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[11]  D. Roberts,et al.  Limitations of the Score‐Difference Method in Detecting Cheating in Recognition Test Situations , 1987 .

[12]  Solomon E. Feldman,et al.  College cheating as a function of subject and situational variables. , 1964 .

[13]  Francis S. Bellezza,et al.  Detection of Cheating on Multiple-Choice Tests by Using Error-Similarity Analysis , 1989 .

[14]  C. Mitchell Dayton,et al.  Improved estimation of academic cheating behavior using the randomized response technique , 1987 .

[15]  Robert B. Frary,et al.  Statistical Detection of Multiple-Choice Answer Copying: Review and Commentary , 1993 .

[16]  R. Linn.  Educational measurement, 3rd ed. , 1989 .

[17]  H. Gulliksen.  Theory of mental tests , 1952 .

[18]  M. Kendall,et al.  Kendall's Advanced Theory of Statistics: Volume 1 Distribution Theory , 1987 .

[19]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[20]  Melody A. Graham,et al.  Cheating at Small Colleges: An Examination of Student and Faculty Attitudes and Behaviors. , 1994 .

[21]  Luz Bay.  Detection of Cheating on Multiple-Choice Examinations. , 1995 .

[22]  R. Darrell Bock,et al.  Estimating item parameters and latent ability when responses are scored in two or more nominal categories , 1972 .