Three Approaches to Using Lengthy Ordinal Scales in Structural Equation Models

Lengthy scales or testlets pose certain challenges for structural equation modeling (SEM) if all the items are included as indicators of a latent construct. Three general approaches to modeling lengthy scales in SEM (parceling, latent scoring, and shortening) have been reviewed and evaluated. A hypothetical population model is simulated containing two exogenous constructs with 14 indicators each and an endogenous construct with four indicators. The simulation generates data sets with varying numbers of response options, two types of distributions, factor loadings ranging from low to high, and sample sizes ranging from small to moderate. The population model is varied to incorporate one of the following: (a) single parcels, (b) various parcels as indicators of two exogenous constructs, (c) latent scores as observed exogenous variables, and (d) four and six individual items as indicators of two exogenous constructs. The dependent variables evaluated are biases in the covariance and partial covariance population parameters. Biases in these parameters are found to be minimal under the following conditions: (a) when parcels of indicators of five response options are used as indicators of two latent exogenous constructs, (b) when latent scores are used as observed variables at sample sizes above 100 and with indicators that are relatively less skewed in the case of dichotomous indicators, and (c) when four or six individual items with high or diverse factor loadings are used as indicators of two exogenous constructs. These findings provide guidelines for resolving the inconsistency of findings from applying various approaches to empirical data.

[1]  Mark Shevlin,et al.  Summated rating scales. A Monte Carlo investigation of the effects of reliability and collinearity in regression models , 1997 .

[2]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[3]  R. Lennox,et al.  Conventional wisdom on measurement: A structural equation perspective. , 1991 .

[4]  James C. Anderson,et al.  STRUCTURAL EQUATION MODELING IN PRACTICE: A REVIEW AND RECOMMENDED TWO-STEP APPROACH , 1988 .

[5]  Thomas D. Fletcher,et al.  The Effects of Parcels and Latent Variable Scores on the Detection of Interactions in Structural Equation Modeling , 2005 .

[6]  Louis Guttman,et al.  THE DETERMINACY OF FACTOR SCORE MATRICES WITH IMPLICATIONS FOR FIVE OTHER BASIC PROBLEMS OF COMMON‐FACTOR THEORY1 , 1955 .

[7]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[8]  R P McDonald,et al.  Latent Traits and the Possibility of Motion. , 1996, Multivariate behavioral research.

[9]  H W Marsh,et al.  Is More Ever Too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis. , 1998, Multivariate behavioral research.

[10]  Patrick S Malone,et al.  A 12-Year Prospective Study of Patterns of Social Information Processing Problems and Externalizing Behaviors , 2006, Journal of abnormal child psychology.

[11]  T. Little,et al.  To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits , 2002 .

[12]  Rosalie J. Hall,et al.  Item Parceling Strategies in SEM: Investigating the Subtle Effects of Unmodeled Secondary Constructs , 1999 .

[13]  Willem E. Saris,et al.  Alternative approaches to structural modeling of ordinal data: A Monte Carlo study , 1997 .

[14]  Scott L. Hershberger,et al.  The New Rules of Measurement : What Every Psychologist and Educator Should Know , 1999 .

[15]  Raymond B. Cattell,et al.  The Radial Parcel Double Factoring Design: A Solution to the Item-Vs-Parcel Controversy , 1975 .

[16]  G. Goude,et al.  On fundamental measurement in psychology , 1962 .

[17]  Daniel A. Sass,et al.  The Effects of Parceling Unidimensional Scales on Structural Parameter Estimates in Structural Equation Modeling , 2006 .

[18]  B. Muthén,et al.  How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power , 2002 .

[19]  T. Achenbach The Child Behavior Profile: I. Boys aged 6--11. , 1978, Journal of consulting and clinical psychology.

[20]  Tamara Halle,et al.  Scaling Back Survey Scales , 2002 .

[21]  D. Bandalos The Effects of Item Parceling on Goodness-of-Fit and Parameter Estimate Bias in Structural Equation Modeling , 2002 .

[22]  Herbert W. Marsh,et al.  Self Description Questionnaire III: The construct validity of multidimensional self-concept ratings by late adolescents. , 1984 .

[23]  Fadia Nasser,et al.  A Monte Carlo Study Investigating the Impact of Item Parceling on Measures of Fit in Confirmatory Factor Analysis , 2003 .

[24]  G. Freytag [CORRELATION AND CAUSALITY]. , 1964, Psychiatrie, Neurologie, und medizinische Psychologie.

[25]  S. Embretson,et al.  Item response theory for psychologists , 2000 .

[26]  F. Samejima Estimation of latent ability using a response pattern of graded scores , 1968 .

[27]  Keith F. Widaman,et al.  Unidimensional Versus Domain Representative Parceling of Questionnaire Items: An Empirical Example , 1994 .

[28]  T. Little,et al.  On selecting indicators for multivariate measurement and modeling with latent variables: When "good" indicators are bad and "bad" indicators are good. , 1999 .

[29]  R. Cattell,et al.  Validation and intensification of the Sixteen Personality Factor Questionnaire. , 1956, Journal of clinical psychology.

[30]  H. Marsh,et al.  The use of item parcels in structural equation modelling: non-normal data and small sample sizes. , 2004, The British journal of mathematical and statistical psychology.

[31]  Kenneth A. Bollen,et al.  Monte Carlo Experiments: Design and Implementation , 2001 .

[32]  S. Reise,et al.  Parameter Recovery in the Graded Response Model Using MULTILOG , 1990 .

[33]  Pere J. Ferrando,et al.  Difficulty, Discrimination, and Information Indices in the Linear Factor Analysis Model for Continuous Item Responses , 2009 .

[34]  B. Muthén A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators , 1984 .

[35]  J. Wisenbaker,et al.  A Monte Carlo Study Investigating the Impact of Item Parceling Strategies on Parameter Estimates and Their Standard Errors in CFA , 2006 .

[36]  Michael D Slater,et al.  Brief measures of sensation seeking for screening and large-scale surveys. , 2003, Drug and alcohol dependence.

[37]  Jan de Leeuw,et al.  On the relationship between item response theory and factor analysis of discretized variables , 1987 .

[38]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model , 1992 .

[39]  Susan E. Embretson,et al.  Item Response Theory Models and Spurious Interaction Effects in Factorial ANOVA Designs , 1996 .

[40]  S. Finney,et al.  Item parceling issues in structural equation modeling , 2001 .

[41]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables , 1985 .