Latent class multitrait-multimethod models

The present paper suggests a statistical method, the latent class MTMM model, of estimating the quality of single questions while making fewer assumptions than have been made so far in such evaluations. The method is a combination of the multitrait-multimethod research design of Campbell and Fiske (1959), the basic response model for single questions of Saris and Andrews (1991), and the latent class factor model of Vermunt et al. (2004). The latent class MTMM model is thus not novel in itself, but combines an existing design, model, and method to improve the analysis of single questions in survey research. A real experiment from the European Social Survey (ESS) is analyzed and the results are discussed at length, yielding valuable insights into the functioning of these questions.

[1]  S. Schwartz.  Universals in the Content and Structure of Values: Theoretical Advances and Empirical Tests in 20 Countries , 1992 .

[2]  Examination of the Relationship between Nonresponse and Measurement Error in a Validation Study of Alumni , 2006 .

[3]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[4]  Richard Goldstein.  Latent Class and Discrete Latent Trait Models: Similarities and Differences , 1998 .

[5]  Willem E. Saris,et al.  Variation in response functions : a source of measurement error in attitude research , 1988 .

[6]  D. Marlowe,et al.  A new scale of social desirability independent of psychopathology. , 1960, Journal of consulting psychology.

[7]  F. Vijver,et al.  Cross-Cultural Survey Methods , 2002 .

[8]  Willem E. Saris,et al.  Evaluation of Measurement Instruments Using a Structural Modeling Approach , 2011 .

[9]  Willem E. Saris,et al.  Fit of Different Models for Multitrait-Multimethod Experiments , 2002 .

[10]  Harry C. Triandis,et al.  Effects of Culture and Response Format on Extreme Response Style , 1989 .

[11]  J. Krosnick,et al.  Comparing questions with agree/ disagree response options to questions with item-specific response options , 2010 .

[12]  S. Haberman.  Analysis of qualitative data , 1978 .

[13]  Willem E. Saris,et al.  Design, Evaluation, and Analysis of Questionnaires for Survey Research , 2007 .

[14]  John R. Donoghue,et al.  An Empirical Examination of the IRT Information of Polytomously Scored Reading Items Under the Generalized Partial Credit Model , 1994 .

[15]  Jeroen K. Vermunt,et al.  Factor Analysis with Categorical Indicators: A Comparison Between Traditional and Latent Class Approaches , 2005 .

[16]  Jürgen Rost,et al.  Multimethod Item Response Theory. , 2006 .

[17]  A. Pollice,et al.  A Generalization of the Polychoric Correlation Coefficient , 2006 .

[18]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[19]  H. Joe.  Asymptotic efficiency of the two-stage estimation method for copula-based models , 2005 .

[20]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .

[21]  Neil Henry.  Latent structure analysis , 1969 .

[22]  Willem E. Saris,et al.  8. A New Approach to Evaluating the Quality of Measurement Instruments: The Split-Ballot MTMM Design , 2004 .

[23]  Peter Lynn,et al.  How representative can a multi-nation survey be? , 2007 .

[24]  J. Hagenaars,et al.  Applied Latent Class Analysis , 2003 .

[25]  Annette Scherpenzeel,et al.  The Validity and Reliability of Survey Questions , 1997 .

[26]  Jay Magidson,et al.  Latent Class Factor and Cluster Models, Bi-Plots, and Related Graphical Displays , 2001 .

[27]  Jaak Billiet,et al.  Modeling Acquiescence in Measurement Models for Two Balanced Sets of Items , 2000 .

[28]  Phil Edwards,et al.  Survey Errors and Survey Costs , 1991 .

[29]  W. M. Veld.  The Survey Response Dissected. A new theory about the surrvey response process , 2003 .

[30]  Wayne A. Fuller,et al.  Measurement Error Models , 1988 .

[31]  D. Jackson,et al.  Content and style in personality assessment. , 2011, Psychological bulletin.

[32]  Paul F. Lazarsfeld,et al.  Latent Structure Analysis. , 1969 .

[33]  J. Vermunt,et al.  Latent Gold 4.0 User's Guide , 2005 .

[34]  Willem E. Saris,et al.  Estimation of the effects of measurement characteristics on the quality of survey questions , 2007 .

[35]  P. Marsden.  Margins of Error: A Study of Reliability in Survey Measurement , 2011 .

[36]  W. Grove,et al.  A latent trait finite mixture model for the analysis of rating agreement. , 1993, Biometrics.

[37]  J. Krosnick.  Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[38]  Alan Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[39]  L. A. Goodman.  Exploratory latent structure analysis using both identifiable and unidentifiable models , 1974 .

[40]  B. Muthén.  A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators , 1984 .

[41]  Forrest W. Young,et al.  Nonmetric individual differences multidimensional scaling: An alternating least squares method with optimal scaling features , 1977 .

[42]  F. M. Andrews.  Construct Validity and Error Components of Survey Measures: A Structural Modeling Approach , 1984 .

[43]  Neal Schmitt,et al.  Methodology Review: Analysis of Multitrait-Multimethod Matrices , 1986 .

[44]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[45]  David Thissen,et al.  A taxonomy of item response models , 1986 .

[46]  R. Groves,et al.  Survey Errors and Survey Costs. , 1991 .

[47]  E. Muraki.  Information Functions of the Generalized Partial Credit Model , 1993 .

[48]  Paul Biemer,et al.  On the validity of Markov latent class analysis for estimating classification error in labor force data , 1999 .