Fitting Item Response Theory Models to Two Personality Inventories: Issues and Insights

The present study compared the fit of several IRT models to two personality assessment instruments. Data from 13,059 individuals responding to the US-English version of the Fifth Edition of the Sixteen Personality Factor Questionnaire (16PF) and 1,770 individuals responding to Goldberg's 50 item Big Five Personality measure were analyzed. Various issues pertaining to the fit of the IRT models to personality data were considered. We examined two of the most popular parametric models designed for dichotomously scored items (i.e., the two- and three-parameter logistic models) and a parametric model for polytomous items (Samejima's graded response model). Also examined were Levine's nonparametric maximum likelihood formula scoring models for dichotomous and polytomous data, which were previously found to provide good fits to several cognitive ability tests (Drasgow, Levine, Tsien, Williams, & Mead, 1995). The two- and three-parameter logistic models fit some scales reasonably well but not others; the graded response model generally did not fit well. The nonparametric formula scoring models provided the best fit of the models considered. Several implications of these findings for personality measurement and personnel selection were described.

[1]  D. Cooke,et al.  An Item Response Theory Analysis of the Hare Psychopathy Checklist-Revised , 1997 .

[2]  F. Drasgow,et al.  Effects of the testing situation on item responding: cause for concern. , 2001, The Journal of applied psychology.

[3]  F. Drasgow,et al.  Toward a theory of individual differences and leadership: understanding the motivation to lead. , 2001, The Journal of applied psychology.

[4]  Chet Robie,et al.  Modeling faking good on personality items: An item-level analysis. , 1999 .

[5]  R. P. McDonald,et al.  Test Theory: A Unified Treatment , 1999 .

[6]  J. Butcher,et al.  Advances in clinical personality measurement : An item response theory analysis of the MMPI-2 PSY-5 scales , 1999 .

[7]  Douglas H. Wedell,et al.  Validity Issues in the Likert and Thurstone Approaches to Attitude Measurement , 1999 .

[8]  L. R. Goldberg.  A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models , 1999 .

[9]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[10]  J. Bermúdez,et al.  Personality Psychology in Europe , 1997 .

[11]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[12]  David Andrich,et al.  A hyperbolic cosine latent trait model for unfolding polytomous responses: Reconciling Thurstone and Likert methodologies , 1996 .

[13]  Niels G. Waller,et al.  Exploring nonlinear models in personality assessment: Development and preliminary validation of a negative emotionality scale. , 1996 .

[14]  J. Huisman.  The Netherlands , 1996, The Lancet.

[15]  Fritz Drasgow,et al.  Detecting Faking on a Personality Instrument Using Appropriateness Measurement , 1996 .

[16]  Robert D. Hare,et al.  The Hare Psychopathy Checklist-Revised , 1996 .

[17]  Kevin R. Murphy,et al.  Individual differences and behavior in organizations , 1996 .

[18]  Fritz Drasgow,et al.  Fitting Polytomous Item Response Theory Models to Multiple-Choice Tests , 1995 .

[19]  A. Feingold,et al.  Gender differences in personality: a meta-analysis. , 1994, Psychological bulletin.

[20]  A. R. Harkness,et al.  The Personality Psychopathology Five (PSY-5): Issues from the pages of a diagnostic manual instead of a dictionary. , 1994 .

[21]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[22]  F. Schmidt,et al.  Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance. , 1993 .

[23]  H. Kimmel,et al.  An Item Response Theory Evaluation of an English Version of the Trier Personality Inventory (TPI) , 1993 .

[24]  J. Mckillip,et al.  Fundamentals of item response theory , 1993 .

[25]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[26]  Murray R. Barrick,et al.  Autonomy as a moderator of the relationships between the Big Five personality dimensions and job performance. , 1993 .

[27]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[28]  Murray R. Barrick,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS , 1991 .

[29]  R. Hogan.  Personality and personality measurement. , 1991 .

[30]  Newell K. Eaton,et al.  Criterion-related validities of personality constructs and the effect of response distortion on those validities , 1990 .

[31]  Norman G. Peterson,et al.  PROJECT A: SPECIFICATION OF THE PREDICTOR DOMAIN AND DEVELOPMENT OF NEW SELECTION/ CLASSIFICATION TESTS , 1990 .

[32]  S. Reise,et al.  Fitting the Two-Parameter Model to Personality Data , 1990 .

[33]  Eiji Muraki,et al.  Fitting a Polytomous Item Response Model to Likert-Type Data , 1990 .

[34]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[35]  P. Bobko,et al.  Variance Homogeneity in Interactive Regression A Clarifying Note About Data Transformations , 1990 .

[36]  Paul R. Sackett,et al.  INTEGRITY TESTING FOR PERSONNEL SELECTION: AN UPDATE , 1989 .

[37]  Cees A. W. Glas,et al.  The derivation of some tests for the rasch model from the multinomial distribution , 1988 .

[38]  S. Zedeck,et al.  Relations Between Measures of Typical and Maximum Job Performance , 1988 .

[39]  David Andrich,et al.  The Application of an Unfolding Model of the PIRT Type to the Measurement of Attitude , 1988 .

[40]  William Stout,et al.  A nonparametric approach for assessing latent trait unidimensionality , 1987 .

[41]  Paul Jansen,et al.  Latent trait models and dichotomization of graded responses , 1986 .

[42]  John Hattie,et al.  Methodology Review: Assessing Unidimensionality of Tests and ltenls , 1985 .

[43]  David Thissen,et al.  A response model for multiple choice items , 1984 .

[44]  Michael V. LeVine.  An Introduction to Multilinear Formula Score Theory. , 1984 .

[45]  George J. Huba,et al.  Structures of psychological distress: Testing confirmatory hierarchical models. , 1984 .

[46]  John Hattie,et al.  An Empirical Study of Various Indices for Determining Unidimensionality. , 1984, Multivariate behavioral research.

[47]  David Thissen,et al.  A Response Model for Multiple Choice Items. Psychometric Technical Report No. 1. , 1983 .

[48]  F. Drasgow,et al.  Modified parallel analysis: A procedure for examining the latent dimensionality of dichotomously scored item responses. , 1983 .

[49]  Arnold L. van den Wollenberg,et al.  Two new test statistics for the rasch model , 1982 .

[50]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[51]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[52]  J. H. Steiger.  Statistically based tests for the number of common factors , 1980 .

[53]  A New Family of Models for the Multiple-Choice Item. , 1979 .

[54]  Richard G. Montanelli,et al.  An Investigation of the Parallel Analysis Criterion for Determining the Number of Common Factors , 1975 .

[55]  R. Darrell Bock,et al.  Estimating item parameters and latent ability when responses are scored in two or more nominal categories , 1972 .

[56]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[57]  C. Coombs.  A theory of data. , 1965, Psychological review.