The person response function as a tool in person-fit research

Item responses that do not fit an item response theory (IRT) model may cause the latent trait value to be inaccurately estimated. In the past two decades several statistics have been proposed that can be used to identify nonfitting item score patterns. These statistics all yieldscalar values. Here, the use of the person response function (PRF) for identifying nonfitting item score patterns was investigated. The PRF is afunction and can be used for diagnostic purposes. First, the PRF is defined in a class of IRT models that imply an invariant item ordering. Second, a person-fit method proposed by Trabin & Weiss (1983) is reformulated in a nonparametric IRT context assuming invariant item ordering, and statistical theory proposed by Rosenbaum (1987a) is adapted to test locally whether a PRF is nonincreasing. Third, a simulation study was conducted to compare the use of the PRF with the person-fit statistic ZU3. It is concluded that the PRF can be used as a diagnostic tool in person-fit research.

[1]  Charles Lewis,et al.  A Nonparametric Approach to the Analysis of Dichotomous Item Responses , 1982 .

[2]  Klaas Sijtsma,et al.  Methodology Review: Nonparametric IRT Approaches to the Analysis of Dichotomous Item Scores , 1998 .

[3]  Paul R. Rosenbaum,et al.  Probability inequalities for latent scales , 1987 .

[4]  B. Junker.  Conditional association, essential independence and monotone unidimensional Item response models , 1993 .

[5]  D. S. Sivia,et al.  Data Analysis , 1996, Encyclopedia of Evolutionary Psychological Science.

[6]  K. C. Klauer.  Exact and best confidence intervals for the ability parameter of the Rasch model , 1991 .

[7]  J. Ramsay.  Kernel smoothing approaches to nonparametric item characteristic curve estimation , 1991 .

[8]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[9]  F. Lord.  A theory of test scores. , 1952 .

[10]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[11]  P. Rosenbaum.  Comparing item characteristic curves , 1987 .

[12]  Herbert Hoijtink,et al.  The many null distributions of person fit indices , 1990 .

[13]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[14]  K Sijtsma,et al.  A survey of theory and methods of invariant item ordering. , 1996, The British journal of mathematical and statistical psychology.

[15]  Donald B. Rubin,et al.  Measuring the Appropriateness of Multiple-Choice Test Scores , 1979 .

[16]  Michael V. LeVine,et al.  Appropriateness measurement: Review, critique and validating studies , 1982 .

[17]  J. Ramsay.  A similarity-based smoothing approach to nondimensional item analysis , 1995 .

[18]  H. V. D. Flier,et al.  Deviant Response Patterns and Comparability of Test Scores , 1982 .

[19]  D. Grayson,et al.  Two-group classification in latent trait theory: Scores with monotone likelihood ratio , 1988 .

[20]  Fritz Drasgow,et al.  Detecting Inappropriate Test Scores with Optimal and Practical Appropriateness Indices , 1987 .

[21]  Hartmann Scheiblechner,et al.  Isotonic ordinal probabilistic models (ISOP) , 1995 .

[22]  Patrick Suppes,et al.  When are Probabilistic Explanations Possible , 1981 .

[23]  Klaas Sijtsma,et al.  Detection of Aberrant Item Score Patterns: A Review of Recent Developments. Research Report 94-8. , 1994 .

[24]  Karl Christoph Klauer,et al.  An approximately standardized person test for assessing consistency with a latent trait model , 1990 .

[25]  David J. Weiss.  The Stratified Adaptive Computerized Ability Test. , 1973 .

[26]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[27]  K. C. Klauer.  An exact and optimal standardized person test for assessing consistency with the rasch model , 1991 .

[28]  Klaas Sijtsma,et al.  Influence of Test and Person Characteristics on Nonparametric Appropriateness Measurement , 1994 .

[29]  P. Rosenbaum,et al.  Conditional Association and Unidimensionality in Monotone Latent Variable Models , 1985 .

[30]  R. Hambleton,et al.  Item Response Theory: Principles and Applications , 1984 .

[31]  James Lumsden.  Tests are perfectly reliable , 1978 .

[32]  Jules L. Ellis,et al.  Local homogeneity in latent trait models. A characterization of the homogeneous monotone irt model , 1993 .

[33]  Huynh Huynh,et al.  A new proof for monotone likelihood ratio for the sum of independent bernoulli random variables , 1994 .

[34]  Brian W. Junker,et al.  Stochastic ordering using the latent trait and the sum score in polytomous IRT models , 1997 .

[35]  David J. Weiss,et al.  The Person Response Curve: Fit of Individuals to Item Response Theory Models , 1983 .

[36]  Fritz Drasgow,et al.  Optimal Identification of Mismeasured Individuals. , 1996 .

[37]  Klaas Sijtsma,et al.  Reliability of test scores in nonparametric item response theory , 1987 .

[38]  Rob R. Meijer,et al.  Person-Fit Research: An Introduction , 1996 .

[39]  Rob R. Meijer,et al.  A Comparison of the Person Response Function and the lz Person-Fit Statistic , 1998 .

[40]  William Stout,et al.  A New Item Response Theory Modeling Approach with Applications to Unidimensionality Assessment and Ability Estimation , 1990 .