Scoring Method and the Detection of Person Misfit in a Personality Assessment Context

The purpose of this research was to explore psycho metric issues pertinent to the application of an IRT based person-fit (response aberrancy) detection statistic in the personality measurement domain. Monte carlo data analyses were conducted to address issues regarding the l z person-fit statistic. The major issues explored were characteristics of the null distribution of l z and its power to identify nonfitting response patterns under different scoring strategies. There were two main results. First, the l z index null distribution was not well standardized when item parameters of personality scales were used; the l z null distribution variance was significantly less than the hypothesized value of 1.0 under several conditions. Second, the power of l z to detect response misfit was affected by the scoring method. Detection power was optimal when a biweight estimator of θ was used. Recommendations are made regarding proper implementation of person-fit statistics in personality measurement.

[1]  David J. Whitney,et al.  Appropriateness Fit and Criterion-Related Validity , 1993 .

[2]  Fritz Drasgow,et al.  Appropriateness Measurement for Some Multidimensional Test Batteries , 1991 .

[3]  Steven P. Reise,et al.  The Influence of Test Characteristics on the Detection of Aberrant Response Patterns , 1991 .

[4]  R. D. Bock,et al.  Adaptive EAP Estimation of Ability in a Microcomputer Environment , 1982 .

[5]  Robert J. Mislevy,et al.  Biweight Estimates of Latent Ability , 1982 .

[6]  Howard Wainer,et al.  Estimating Ability With the Wrong Model , 1987 .

[7]  Fritz Drasgow,et al.  Detecting Inappropriate Test Scores with Optimal and Practical Appropriateness Indices , 1987 .

[8]  Menucha Birenbaum,et al.  Effect of Dissimulation Motivation and Anxiety on Response Pattern Appropriateness Measures , 1986 .

[9]  S. Reise,et al.  Fitting the Two-Parameter Model to Personality Data , 1990 .

[10]  Steven P. Reise,et al.  Traitedness and the assessment of response pattern scalability , 1993 .

[11]  David J. Weiss,et al.  Improving Measurement Quality and Efficiency with Adaptive Testing , 1982 .

[12]  Fritz Drasgow,et al.  Appropriateness measurement with polychotomous item response models and standardized indices , 1984 .

[13]  Fritz Drasgow,et al.  Choice of Test Model for Appropriateness Measurement , 1982 .

[14]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[15]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .