Accuracy of Person-Fit Statistics

Using a Monte Carlo experimental design, this research examined the relationship between answer patterns’ aberrance rates and person-fit statistics (PFS) accuracy. It was observed that as the aberrance rate increased, the detection rates of PFS also increased until, in some situations, a peak was reached and then the detection rates of PFS decreased with increases in aberrance rates. Furthermore, the results suggest that ECI2Z was somewhat more robust to high levels of aberrance than lz , HT , and U3 when cheating was simulated. The results of this study shed light on a limitation of PFS analysis.

[1]  Frank B. Baker,et al.  Item Response Theory : Parameter Estimation Techniques, Second Edition , 2004 .

[2]  Donald B. Rubin,et al.  Measuring the Appropriateness of Multiple-Choice Test Scores , 1979 .

[3]  Seung W. Choi.  PERSONz: Person Misfit Detection Using the l z Statistic and Monte Carlo Simulations , 2010 .

[4]  Kikumi K. Tatsuoka,et al.  Caution indices based on item response theory , 1984 .

[5]  George Karabatsos,et al.  Comparing the Aberrant Response Detection Performance of Thirty-Six Person-Fit Statistics , 2003 .

[6]  Fritz Drasgow,et al.  Detecting Inappropriate Test Scores with Optimal and Practical Appropriateness Indices , 1987 .

[7]  Stephen Olejnik,et al.  The Power of Rasch Person-Fit Statistics in Detecting Unusual Response Patterns , 1997 .

[8]  Fritz Drasgow,et al.  Appropriateness Measurement: Validating Studies and Variable Ability Models , 1983 .

[9]  Cornelis A.W. Glas,et al.  A Bayesian Approach to Person Fit Analysis in Item Response Theory Models , 2003 .

[10]  C. St-Onge,et al.  A Monte Carlo Study of the Effect of Item Characteristic Curve Estimation on the Accuracy of Three Person-Fit Statistics , 2009 .

[11]  W. Emons.  Nonparametric Person-Fit Analysis of Polytomous Item Scores , 2008 .

[12]  Klaas Sijtsma,et al.  Detection of Aberrant Item Score Patterns: A Review of Recent Developments. Research Report 94-8. , 1994 .

[13]  Tom A. B. Snijders,et al.  Asymptotic null distribution of person fit statistics with estimated person parameter , 2001 .

[14]  Ronald K. Hambleton,et al.  Applications of Item Response Theory , 1983 .

[15]  Lawrence M. Rudner.  INDIVIDUAL ASSESSMENT ACCURACY , 1983 .

[16]  Rosemary Baker,et al.  Item response theory , 1985 .

[17]  David J. Weiss,et al.  Book Review : New Horizons in Testing: Latent Trait Test Theory and Computerized Adaptive Testing David J. Weiss (Ed.) New York: Academic Press, 1983, 345 pp., $35.00 , 1984 .

[18]  K. Sijtsma,et al.  Person Fit in Order-Restricted Latent Class Models , 2003 .

[19]  R. Meijer.  Consistency of test behaviour and individual difference in precision of prediction. , 1998 .

[20]  Fritz Drasgow,et al.  Choice of Test Model for Appropriateness Measurement , 1982 .

[21]  Rob R. Meijer,et al.  The Number of Guttman Errors as a Simple and Powerful Person-Fit Statistic , 1994 .

[22]  Tom A. B. Snijders.  Asymptotic distribution of person fit statistics with estimated person parameters , 2001 .

[23]  Rob R. Meijer,et al.  The Influence of the Presence of Deviant Item Score Patterns on the Power of a Person-Fit Statistic , 1994 .

[24]  Menucha Birenbaum.  Comparing the Effectiveness of Several Irt Based Appropriateness Measures in Detecting Unusual Response Patterns , 1985 .

[25]  Fritz Drasgow,et al.  Optimal Identification of Mismeasured Individuals. , 1996 .

[26]  Benjamin D. Wright,et al.  Solving measurement problems with the Rasch model. , 1977 .

[27]  Lawrence M. Rudner,et al.  Person-Fit Statistics: High Potential and Many Unanswered Questions. ERIC/TM Digest. , 1992 .

[28]  Fritz Drasgow,et al.  Appropriateness Measurement for Some Multidimensional Test Batteries , 1991 .

[29]  Christine E. DeMars,et al.  Item Response Theory , 2010, Assessing Measurement Invariance for Applied Research.

[30]  Michael V. LeVine,et al.  Appropriateness measurement: Review, critique and validating studies , 1982 .

[31]  Marc E. Gessaroli,et al.  The Effect of Test Length and IRT Model on the Distribution and Stability of Three Appropriateness Indexes , 1992 .

[32]  H. V. D. Flier,et al.  Deviant Response Patterns and Comparability of Test Scores , 1982 .

[33]  Fritz Drasgow,et al.  Appropriateness measurement with polychotomous item response models and standardized indices , 1984 .

[34]  Klaas Sijtsma,et al.  Influence of Test and Person Characteristics on Nonparametric Appropriateness Measurement , 1994 .

[35]  Klaas Sijtsma,et al.  A Method for Investigating the Intersection of Item Response Functions in Mokken's Nonparametric IRT Model , 1992 .

[36]  Klaas Sijtsma,et al.  Methodology Review: Evaluating Person Fit , 2001 .

[37]  Klaas Sijtsma,et al.  Comparing Simulated and Theoretical Sampling Distributions of the U3 Person-Fit Statistic , 2002 .

[38]  Rob R. Meijer,et al.  A Comparison of the Person Response Function and the lz Person-Fit Statistic , 1998 .

[39]  Klaas Sijtsma,et al.  Testing Hypotheses About the Person-Response Function in Person-Fit Analysis , 2004, Multivariate behavioral research.