Sample Size Requirements for Estimation of Item Parameters in the Multidimensional Graded Response Model

Likert types of rating scales in which a respondent chooses a response from an ordered set of response options are used to measure a wide variety of psychological, educational, and medical outcome variables. The most appropriate item response theory model for analyzing and scoring these instruments when they provide scores on multiple scales is the multidimensional graded response model (MGRM) A simulation study was conducted to investigate the variables that might affect item parameter recovery for the MGRM. Data were generated based on different sample sizes, test lengths, and scale intercorrelations. Parameter estimates were obtained through the flexMIRT software. The quality of parameter recovery was assessed by the correlation between true and estimated parameters as well as bias and root-mean-square-error. Results indicated that for the vast majority of cases studied a sample size of N = 500 provided accurate parameter estimates, except for tests with 240 items when 1000 examinees were necessary to obtain accurate parameter estimates. Increasing sample size beyond N = 1000 did not increase the accuracy of MGRM parameter estimates.

[1]  Lan Yu,et al.  Adult Attachment Ratings (AAR): An Item Response Theory Analysis , 2014, Journal of personality assessment.

[2]  Steven W. Nydick,et al.  Comparing Two Algorithms for Calibrating the Restricted Non-Compensatory Multidimensional IRT Model , 2015, Applied psychological measurement.

[3]  Michael L. Nering,et al.  Handbook of Polytomous Item Response Theory Models , 2010 .

[4]  Li Cai,et al.  HIGH-DIMENSIONAL EXPLORATORY ITEM FACTOR ANALYSIS BY A METROPOLIS–HASTINGS ROBBINS–MONRO ALGORITHM , 2010 .

[5]  Alberto Maydeu-Olivares,et al.  Estimation of IRT graded response models: limited versus full information methods. , 2009, Psychological methods.

[6]  S. Reise,et al.  Parameter Recovery in the Graded Response Model Using MULTILOG , 1990 .

[7]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[8]  Kelly A. Brennan,et al.  An item response theory analysis of self-report measures of adult attachment. , 2000, Journal of personality and social psychology.

[9]  Alberto Maydeu-Olivares,et al.  Factor Analysis with Ordinal Indicators: A Monte Carlo Study Comparing DWLS and ULS Estimation , 2009 .

[10]  Kimberly S. Maier,et al.  Using a Multivariate Multilevel Polytomous Item Response Theory Model to Study Parallel Processes of Change: The Dynamic Association Between Adolescents' Social Isolation and Engagement With Delinquent Peers in the National Youth Survey , 2010, Multivariate behavioral research.

[11]  G. Vilagut,et al.  Multidimensional item response theory models yielded good fit and reliable scores for the Short Form-12 questionnaire. , 2013, Journal of clinical epidemiology.

[12]  Gregory G. Brown,et al.  Journal of Clinical and Experimental Neuropsychology Parallel Psychometric and Cognitive Modeling Analyses of the Penn Face Memory Test in the Army Study to Assess Risk and Resilience in Servicemembers , 2022 .

[13]  Christine E. DeMars,et al.  Item Response Theory , 2010, Assessing Measurement Invariance for Applied Research.

[14]  Chun Wang,et al.  On Latent Trait Estimation in Multidimensional Compensatory Item Response Models , 2015, Psychometrika.

[15]  M. Kosinski,et al.  Calibration of an item pool for assessing the burden of headaches: An application of item response theory to the Headache Impact Test (HIT™) , 2003, Quality of Life Research.

[16]  Chet Robie,et al.  Modeling faking good on personality items: An item-level analysis. , 1999 .

[17]  D. Bolt,et al.  A multigroup item response theory analysis of the psychopathy checklist--revised. , 2004, Psychological assessment.

[18]  Stanley J. Stough,et al.  Analysis of the Reliability of the Leadership Practices Inventory in the Item Response Theory Framework , 2006 .

[19]  Estimating a Noncompensatory IRT Model Using Metropolis Within Gibbs Sampling , 2011 .

[20]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1969 .

[21]  R. J. Ayala.  The Influence of Multidimensionality on the Graded Response Model. , 1994 .

[22]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[23]  John Hattie,et al.  An examination of the psychometric properties of the physical self-description questionnaire using a polytomous item response model , 2004 .

[24]  R. Krueger,et al.  Capturing abnormal personality with normal personality inventories: an item response theory approach. , 2008, Journal of personality.

[25]  T. A. Warm.  Weighted likelihood estimation of ability in item response theory , 1989 .

[26]  Pere J. Ferrando,et al.  The construct of sensation seeking as measured by Zuckerman's SSS-V and Arnett's AISS: a structural equation model , 2001 .

[27]  Charles A. Scherbaum,et al.  Measuring General Self-Efficacy , 2006 .