Response Styles in Rating Scales

Heterogeneity in response styles can affect the conclusions drawn from rating scale data. In particular, biased estimates can be expected if one ignores a tendency to middle categories or to extreme categories. An adjacent categories model is proposed that simultaneously models the content-related effects and the heterogeneity in response styles. By accounting for response styles, it provides a simple remedy for the bias that occurs if the response style is ignored. The model allows to include explanatory variables that have a content-related effect as well as an effect on the response style. A visualization tool is developed that makes the interpretation of effects easily accessible. The proposed model is embedded into the framework of multivariate generalized linear model, which entails that common estimation and inference tools can be used. Existing software can be used to fit the model, which makes it easy to apply.

[1]  Paul H. C. Eilers,et al.  Flexible smoothing with B-splines and penalties , 1996 .

[2]  C. Wild,et al.  Vector Generalized Additive Models , 1996 .

[3]  R. Darrell Bock,et al.  Estimating item parameters and latent ability when responses are scored in two or more nominal categories , 1972 .

[4]  Irvine Clarke,et al.  Extreme response style in cross-cultural research: An empirical investigation. , 2000 .

[5]  Samuel Messick,et al.  PSYCHOLOGY AND METHODOLOGY OF RESPONSE STYLES1 , 1987 .

[6]  G. Tutz.  Compound Regression Models for Ordered Categorical Data , 1989 .

[7]  Minjeong Jeon,et al.  A generalized item response tree model for psychological assessments , 2016, Behavior research methods.

[8]  Lale Khorramdel,et al.  Measuring Response Styles Across the Big Five: A Multiscale Extension of an Approach Using Multinomial Processing Trees , 2014, Multivariate behavioral research.

[9]  G. Marín,et al.  Extreme Response Style and Acquiescence among Hispanics , 1992 .

[10]  Daniel M. Bolt,et al.  Addressing Score Bias and Differential Item Functioning Due to Individual Differences in Response Style , 2009 .

[11]  Irvine Clarke,et al.  Extreme response style in cross‐cultural research , 2001 .

[12]  T. Yee.  The VGAM Package for Categorical Data Analysis , 2010 .

[13]  Patrick J.F. Groenen,et al.  Identifying Response Styles: A Latent-Class Bilinear Multinomial Logit Model , 2010 .

[14]  G. Moors.  Facts and Artefacts in the Comparison of Attitudes Among Ethnic Minorities. A Multigroup Latent Class Structure Model with Adjustment for Response Style Behavior , 2004 .

[15]  G. Moors.  Diagnosing Response Style Behavior by Means of a Latent-Class Factor Approach. Socio-Demographic Correlates of Gender Role Attitudes and Perceptions of Ethnic Discrimination Reexamined , 2003 .

[16]  Gerhard Tutz,et al.  Regression for Categorical Data , 2011 .

[17]  Timothy R. Johnson,et al.  On the use of heterogeneous thresholds ordinal regression models to account for individual differences in response style , 2003 .

[18]  D. Bolt,et al.  Nested Logit Models for Multiple-Choice Item Response Data , 2010 .

[19]  Thorsten Meiser,et al.  Validity of Multiprocess IRT Models for Separating Content and Response Styles , 2014 .

[20]  Hans Baumgartner,et al.  Response Styles in Marketing Research: A Cross-National Investigation , 2001 .

[21]  Michael Eid,et al.  Detecting Measurement Invariance in Organizational Surveys* * The original data upon which this paper is based are available at www.hhpub.com/journals/ejpa , 2000 .

[22]  G. Meisenberg,et al.  Are acquiescent and extreme response styles related to low intelligence and education , 2008 .

[23]  B. Weijters,et al.  The effect of rating scale format on response styles: the number of response categories and response catgory labels , 2010 .

[24]  Eric R. Ziegel,et al.  Multivariate Statistical Modelling Based on Generalized Linear Models , 2002, Technometrics.

[25]  G. Moors,et al.  Ranking the Ratings: A Latent-Class Regression Model to Control for Overall Agreement in Opinion Research , 2010 .

[26]  David E. Wiley,et al.  Improving inquiry in social science : a volume in honor of Lee J. Cronbach , 1991 .

[27]  Paul De Boeck,et al.  IRTrees: Tree-Based Item Response Models of the GLMM Family , 2012 .

[28]  Daniel M. Bolt,et al.  Multiscale Measurement of Extreme Response Style , 2011 .

[29]  Maria Iannario,et al.  Measuring Job Satisfaction with CUB Models , 2013 .

[30]  Ivy Liu,et al.  Analysis of Ordinal Categorical Data, 2nd edn by Alan Agresti , 2011 .

[31]  U. Böckenholt.  Modeling multiple response processes in judgment and choice. , 2012, Psychological methods.

[32]  Y. Poortinga,et al.  Response Styles in Rating Scales , 2004 .

[33]  G. Moors,et al.  Measurement Equivalence in Solidarity Attitudes in Europe , 2009 .

[34]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[35]  Anne Thissen-Roe,et al.  A Two-Decision Model for Responses to Likert-Type Items , 2013 .

[36]  P. McCullagh.  Analysis of Ordinal Categorical Data , 1985 .

[37]  Patrick M. O'Malley,et al.  Yea-Saying, Nay-Saying, and Going to Extremes: Black-White Differences in Response Styles , 1984 .

[38]  Troy Devon Thomas,et al.  Response Styles in Survey Research: A Literature Review of Antecedents, Consequences, and Remedies , 2013 .