Testing the Nonlinearity Assumption Underlying the Use of Reverse-Keyed Items: A Logical Response Perspective

Researchers often assume a strong, linear relationship between regular- and reverse-keyed items, with responses on regular-keyed items (e.g., agree) perfectly mirroring those on reverse-keyed items (e.g., disagree). The current research challenges this received view and propounds a possible nonlinear relationship, partly due to the logical tendency of midlevel respondents to disagree with both types of items. In four examples (reported human height, job satisfaction, positive-negative affect, and self-esteem; total N = 50,544), a nonlinear model consistently explained additional item variance beyond a linear model. We further demonstrate that this relationship is moderated by item characteristics such as item extremity (job satisfaction) and item softening (self-esteem). Suboptimal modeling of the relationship may result in the apparent bidmensionality of a construct that characterizes regular- and reverse-keyed items as separate factors. User-friendly syntax for the examination of nonlinearity is provided to enhance the accessibility of the procedure.

[1]  John P. Meyer,et al.  Why Do People Agree With Both Regular and Reversed Items? A Logical Response Perspective , 2021, Assessment.

[2]  Daniel A. Newman,et al.  Missing Data , 2014 .

[3]  Dylan Molenaar,et al.  The Scale of Positive and Negative Experience (SPANE) , 2020 .

[4]  J. Veilleux,et al.  Affect balance predicts daily emotional experience , 2020 .

[5]  E. Diener,et al.  The manipulation of affect: A meta-analysis of affect induction procedures. , 2020, Psychological bulletin.

[6]  Jacky C. K. Ng,et al.  Latent Moderation Analysis: A Factor Score Approach , 2020, Structural Equation Modeling: A Multidisciplinary Journal.

[7]  M. Alicke,et al.  The better-than-average effect in comparative self-evaluation: A comprehensive review and meta-analysis. , 2019, Psychological bulletin.

[8]  M. Michaelides.  Negative Keying Effects in the Factor Structure of TIMSS 2011 Motivation Scales and Associations with Reading Achievement , 2019, Applied Measurement in Education.

[9]  Leonard J. Simms,et al.  Does the Number of Response Options Matter? Psychometric Perspectives Using Personality Questionnaire Data , 2019, Psychological assessment.

[10]  Murat Yıldırım.  Mediating Role of Resilience in the Relationships Between Fear of Happiness and Affect Balance, Satisfaction With Life, and Flourishing , 2018, Europe's journal of psychology.

[11]  C. Kam.  Testing the Assumption of Population Homogeneity in the Measurement of Dispositional Optimism: Factor Mixture Modeling Analysis , 2018, Journal of personality assessment.

[12]  Xitao Fan,et al.  Investigating Response Heterogeneity in the Context of Positively and Negatively Worded Items by Using Factor Mixture Modeling , 2018, Organizational Research Methods.

[13]  M. Eid,et al.  Analyzing Different Types of Moderated Method Effects in Confirmatory Factor Models for Structurally Different Methods , 2018 .

[14]  Timo Gnambs,et al.  Cognitive Abilities Explain Wording Effects in the Rosenberg Self-Esteem Scale , 2020, Assessment.

[15]  Graham A du Plessis,et al.  Validation of the Scale of Positive and Negative Experience in a South African student sample , 2017 .

[16]  Elke Heise,et al.  Measuring the frequency of emotions—validation of the Scale of Positive and Negative Experience (SPANE) in Germany , 2017, PloS one.

[17]  Charles L. Hulin,et al.  Job Attitudes, Job Satisfaction, and Job Affect: A Century of Continuity and of Change , 2017, The Journal of applied psychology.

[18]  C. Geiser,et al.  Examining Trait × Method Interactions Using Mixture Distribution Multitrait–Multimethod Models , 2017, Structural equation modeling : a multidisciplinary journal.

[19]  Cheng-Hsien Li.  Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares , 2016, Behavior research methods.

[20]  K. Grimm,et al.  Augmenting the Correlated Trait–Correlated Method Model for Multitrait–Multimethod Data , 2016 .

[21]  Veljko Jovanović,et al.  Beyond the PANAS: Incremental validity of the Scale of Positive and Negative Experience (SPANE) in relation to well-being , 2015 .

[22]  Joel T. Nadler,et al.  Stuck in the Middle: The Use and Interpretation of Mid-Points in Items on Questionnaires , 2015, The Journal of general psychology.

[23]  A. Howell,et al.  Measuring and Predicting Student Well-Being: Further Evidence in Support of the Flourishing Scale and the Scale of Positive and Negative Experiences , 2015 .

[24]  John P. Meyer,et al.  How Careless Responding and Acquiescence Response Bias Can Influence Construct Dimensionality , 2015 .

[25]  Robbert Sanderman,et al.  Correction: Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain , 2013, PLoS ONE.

[26]  H. Baumgartner,et al.  Reversed item bias: an integrative model. , 2013, Psychological methods.

[27]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[28]  F. Lucidi,et al.  Method Effects: The Problem With Negatively Versus Positively Keyed Items , 2012, Journal of personality assessment.

[29]  Richard M. Ryan,et al.  Self-Determination Theory and Diminished Functioning , 2011, Personality & social psychology bulletin.

[30]  Chu Kim-prieto,et al.  New Well-being Measures: Short Scales to Assess Flourishing and Positive and Negative Feelings , 2010 .

[31]  H. Marsh,et al.  Longitudinal tests of competing factor structures for the Rosenberg Self-Esteem Scale: traits, ephemeral artifacts, and stable response styles. , 2010, Psychological assessment.

[32]  O. Chernyshenko,et al.  Contextual Performance and the Job Satisfaction–Dissatisfaction Distinction: Examining Artifacts and Utility , 2009 .

[33]  M. Bäckström,et al.  Five-factor inventories have a major general factor related to social desirability which can be reduced by framing items neutrally. , 2009 .

[34]  Jüri Allik,et al.  Simultaneous administration of the Rosenberg Self-Esteem Scale in 53 nations: exploring the universal and culture-specific features of global self-esteem. , 2005, Journal of personality and social psychology.

[35]  T. Judge,et al.  Relationship of core self-evaluations traits--self-esteem, generalized self-efficacy, locus of control, and emotional stability--with job satisfaction and job performance: a meta-analysis. , 2001, The Journal of applied psychology.

[36]  J. M. Carroll,et al.  On the bipolarity of positive and negative affect. , 1999, Psychological bulletin.

[37]  Paul E. Spector.  Job satisfaction: application, assessment, cause, and consequences , 1997 .

[38]  Larry J. Williams,et al.  A Confirmatory Factor Analysis Examination of Reverse Coding Effects in Meyer and Allen's Affective and Continuance Commitment Scales , 1996 .

[39]  Chester A. Schriesheim,et al.  An Exploratory and Confirmatory Factor-Analytic Investigation of Item Wording Effects on the Obtained Factor Structures of Survey Questionnaire Measures , 1995 .

[40]  Michael W. Bridges,et al.  Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): a reevaluation of the Life Orientation Test. , 1994, Journal of personality and social psychology.

[41]  John P. Meyer,et al.  Commitment to organizations and occupations: Extension and test of a three-component conceptualization. , 1993 .

[42]  Chester A. Schriesheim,et al.  The Effect of Negation and Polar Opposite Item Reversals on Questionnaire Reliability and Validity: An Experimental Investigation , 1991 .

[43]  Michael T. Brannick,et al.  Construction of a Job in General scale: A comparison of global, composite, and specific measures. , 1989 .

[44]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[45]  Neal Schmitt,et al.  Factors Defined by Negatively Keyed Items: The Result of Careless Respondents? , 1985 .

[46]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[47]  D. J. Lee.  Society and the Adolescent Self-Image , 1969 .

[48]  C. O. Matthews.  The effect of position of printed response words upon children's answers to questions in two-response types of tests. , 1927 .