Detection of Aberrant Item Score Patterns: A Review of Recent Developments. Research Report 94-8.

Methods for detecting item score patterns that are unlikely (aberrant) given that a parametric item response theory (IRT) model gives an adequate description of the data or given the responses of the other persons in the group are discussed. The emphasis here is on the latter group of statistics. These statistics can be applied when a nonparametric model is used to fit the data or when the data are described in the absence of an IRT model. After discussion of the literature on person-fit methods, the use of person-fit statistics in empirical data analysis is briefly discussed. In some situations, the analysis of item score patterns might reveal more information about examinees than the analysis of test scores. Finding an aberrant pattern does not explain the reason for the aberrance. A full person-fit analysis requires additional research into the motives, strategies, and backgrounds of the examinees who deviate from the statistical norm set by the model or group.

[1]  Charles Lewis,et al.  A Nonparametric Approach to the Analysis of Dichotomous Item Responses , 1982 .

[2]  Fritz Drasgow,et al.  Appropriateness measurement with polychotomous item response models and standardized indices , 1984 .

[3]  Fritz Drasgow,et al.  Appropriateness Measurement for Some Multidimensional Test Batteries , 1991 .

[4]  T. Nicolaus Tideman,et al.  Indices of Cheating on Multiple-Choice Tests , 1977 .

[5]  Robert B. Frary,et al.  Statistical Detection of Multiple-Choice Answer Copying: Review and Commentary , 1993 .

[6]  Herbert Hoijtink,et al.  The many null distributions of person fit indices , 1990 .

[7]  Kikumi K. Tatsuoka,et al.  Caution indices based on item response theory , 1984 .

[8]  Fritz Drasgow,et al.  Detecting Inappropriate Test Scores with Optimal and Practical Appropriateness Indices , 1987 .

[9]  Klaas Sijtsma,et al.  Theoretical and Empirical Comparison of the Mokken and the Rasch Approach to IRT , 1990 .

[10]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[11]  Lawrence M. Rudner.  INDIVIDUAL ASSESSMENT ACCURACY , 1983 .

[12]  Klaas Sijtsma,et al.  Influence of Test and Person Characteristics on Nonparametric Appropriateness Measurement , 1994 .

[13]  Benjamin D. Wright,et al.  Solving measurement problems with the Rasch model. , 1977 .

[14]  P. Holland.  When are item response models consistent with observed data? , 1981 .

[15]  R. F. Fagot.  Reliability of Ratings for Multiple Judges: Intraclass Correlation and Metric Scales , 1991 .

[16]  Paul R. Rosenbaum,et al.  Probability inequalities for latent scales , 1987 .

[17]  Michael V. LeVine,et al.  Appropriateness measurement: Review, critique and validating studies , 1982 .

[18]  B. Wright,et al.  Best Test Design. Rasch Measurement. , 1979 .

[19]  W Meredith,et al.  Some results based on a general stochastic model for mental tests , 1965, Psychometrika.

[20]  Karl Christoph Klauer,et al.  An approximately standardized person test for assessing consistency with a latent trait model , 1990 .

[21]  G. Masters,et al.  Rating scale analysis , 1982 .

[22]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[23]  Kikumi K. Tatsuoka,et al.  Spotting Erroneous Rules of Operation by the Individual Consistency Index. , 1983 .

[24]  Donald B. Rubin,et al.  Measuring the Appropriateness of Multiple-Choice Test Scores , 1979 .

[25]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[26]  Steven P. Reise,et al.  The Influence of Test Characteristics on the Detection of Aberrant Response Patterns , 1991 .

[27]  H. V. D. Flier,et al.  Deviant Response Patterns and Comparability of Test Scores , 1982 .

[28]  David J. Whitney,et al.  Appropriateness Fit and Criterion-Related Validity , 1993 .

[29]  Gregory L. Candell,et al.  Modeling Incorrect Responses to Multiple-Choice Items With Multilinear Formula Score Theory , 1989 .

[30]  Klaas Sijtsma,et al.  Mokken Scale Analysis: Theoretical Considerations and an Application to Transitivity Tasks , 1992 .

[31]  M. Liou.  Exact Person Tests for Assessing Model-Data Fit in the Rasch Model. , 1993 .

[32]  R. Hambleton,et al.  Item Response Theory: Principles and Applications , 1984 .

[33]  D. Harnisch.  ITEM RESPONSE PATTERNS: APPLICATIONS FOR EDUCATIONAL PRACTICE , 1983 .

[34]  Kikumi K. Tatsuoka,et al.  A Probabilistic Model for Diagnosing Misconceptions By The Pattern Classification Approach , 1985 .

[35]  William Stout,et al.  A New Item Response Theory Modeling Approach with Applications to Unidimensionality Assessment and Ability Estimation , 1990 .

[36]  Delwyn L. Harnisch,et al.  ANALYSIS OF ITEM RESPONSE PATTERNS. QUESTIONABLE TEST DATA AND DISSIMILAR CURRICULUM PRACTICES , 1981 .

[37]  Fritz Drasgow,et al.  Choice of Test Model for Appropriateness Measurement , 1982 .

[38]  Kikumi K. Tatsuoka,et al.  Detection of Aberrant Response Patterns and their Effect on Dimensionality , 1982 .