Nonparametric person-fit research: Some theoretical issues and an empirical example

In person-fit analysis, it is investigated whether an item score pattern is improbable given the item score patterns of the other persons in the group or given an expected score pattern on the basis of a test model. In this study, several existing group-based statistics are discussed to detect such improbable item score patterns, along with the cut scores that were proposed in the literature to classify an item score pattern as aberrant. By means of a simulation study and an empirical study, the detection rate of these statistics is compared, and the practical use of various cut scores is investigated. It is furthermore demonstrated that person-fit statistics can be used to detect persons with a deficiency of knowledge on an achievement test.

[1]  Rob R. Meijer,et al.  Person-Fit Research: An Introduction , 1996 .

[2]  Klaas Sijtsma,et al.  Detection of Aberrant Item Score Patterns: A Review of Recent Developments. Research Report 94-8. , 1994 .

[3]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[4]  Klaas Sijtsma,et al.  Mokken Scale Analysis: Theoretical Considerations and an Application to Transitivity Tasks , 1992 .

[5]  Herbert Hoijtink,et al.  Person-Fit and the Rasch Model, with an Application to Knowledge of Logical Quantors. , 1996 .

[6]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[7]  Klaas Sijtsma,et al.  Theoretical and Empirical Comparison of the Mokken and the Rasch Approach to IRT , 1990 .

[8]  Charles Lewis,et al.  A Nonparametric Approach to the Analysis of Dichotomous Item Responses , 1982 .

[9]  Kikumi K. Tatsuoka,et al.  Spotting Erroneous Rules of Operation by the Individual Consistency Index. , 1983 .

[10]  Michael V. LeVine,et al.  Appropriateness measurement: Review, critique and validating studies , 1982 .

[11]  B. Wright,et al.  Best Test Design. Rasch Measurement. , 1979 .

[12]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[13]  R. Hambleton,et al.  Item Response Theory: Principles and Applications , 1984 .

[14]  David J. Whitney,et al.  Appropriateness Fit and Criterion-Related Validity , 1993 .

[15]  H. V. D. Flier,et al.  Deviant Response Patterns and Comparability of Test Scores , 1982 .

[16]  D. Harnisch.  ITEM RESPONSE PATTERNS: APPLICATIONS FOR EDUCATIONAL PRACTICE , 1983 .

[17]  David J. Weiss,et al.  The Person Response Curve: Fit of Individuals to Item Response Theory Models , 1983 .

[18]  Robert B. Frary,et al.  Statistical Detection of Multiple-Choice Answer Copying: Review and Commentary , 1993 .

[19]  Rob R. Meijer,et al.  The Number of Guttman Errors as a Simple and Powerful Person-Fit Statistic , 1994 .

[20]  Donald B. Rubin,et al.  Measuring the Appropriateness of Multiple-Choice Test Scores , 1979 .

[21]  Steven P. Reise,et al.  The Influence of Test Characteristics on the Detection of Aberrant Response Patterns , 1991 .

[22]  Delwyn L. Harnisch,et al.  ANALYSIS OF ITEM RESPONSE PATTERNS. QUESTIONABLE TEST DATA AND DISSIMILAR CURRICULUM PRACTICES , 1981 .

[23]  Herbert Hoijtink,et al.  The many null distributions of person fit indices , 1990 .

[24]  Kikumi K. Tatsuoka,et al.  Caution indices based on item response theory , 1984 .

[25]  Fritz Drasgow,et al.  Detecting Inappropriate Test Scores with Optimal and Practical Appropriateness Indices , 1987 .

[26]  M. David Miller.  Time Allocation and Patterns of Item Response. , 1986 .

[27]  William Stout,et al.  A New Item Response Theory Modeling Approach with Applications to Unidimensionality Assessment and Ability Estimation , 1990 .

[28]  Steven P. Reise,et al.  Assessing Person-Fit on Measures of Typical Performance , 1996 .

[29]  P. Holland.  When are item response models consistent with observed data? , 1981 .

[30]  Fritz Drasgow,et al.  Appropriateness measurement with polychotomous item response models and standardized indices , 1984 .

[31]  Klaas Sijtsma,et al.  Influence of Test and Person Characteristics on Nonparametric Appropriateness Measurement , 1994 .