A matter of context: A meta‐analytic investigation of the relative validity of contextualized and noncontextualized personality measures.

The empirical evidence that has accumulated in support of the notion that personality is a valid predictor of employee performance is vast, yet debate on the matter continues. This study investigates frame-of-reference effects as they relate to the validity of self-report measures of personality. Specifically, we compare the validities of general, noncontextualized personality measures and work-specific, contextualized measures. The findings suggest that personality measures are a more valid predictor of performance when the scale items or instructions are framed specifically so as to reference work-specific behaviors. We found that the validities for noncontextualized measures of personality ranged from .02 to .22, with a mean validity of .11. The validities for contextualized measures ranged from .14 to .30, with a mean of .24. Additional moderator analyses were conducted in an effort to examine several alternate explanations for these validity differences. Specifically, we examined differences between the developmental purpose (general use vs. workplace use) and reliabilities of each type of personality measure. We also compared the validities from published studies to those from unpublished studies. Results suggest that these moderators did not have an impact on the validity differences between noncontextualized and contextualized measures.

[1]  Huy Le,et al.  Beyond alpha: an empirical examination of the effects of different sources of measurement error on reliability estimates for measures of individual differences constructs. , 2003, Psychological methods.

[2]  Murray R. Barrick,et al.  Effects of impression management and self-deception on the predictive validity of personality constructs. , 1996, The Journal of applied psychology.

[3]  Karin A. Orvis,et al.  A meta-analytic investigation of conscientiousness in the prediction of job performance: examining the intercorrelations and the incremental validity of narrow traits. , 2006, The Journal of applied psychology.

[4]  Person-environment congruence and perceived work climate , 1988 .

[5]  L. A. Witt,et al.  The Interactive Effects of Extraversion and Conscientiousness on Performance , 2002 .

[6]  Theodore L. Hayes,et al.  Personality correlates of success in total quality manufacturing , 1994 .

[7]  John P. Hausknecht,et al.  Applicant Reactions to Selection Procedures: An Updated Model and Meta-Analysis , 2004 .

[8]  Murray R. Barrick,et al.  An investigation of personality similarity effects (relational and perceived) on peer and supervisor ratings and the role of familiarity and liking , 2001 .

[9]  James R. Van Scotter,et al.  Interpersonal facilitation and job dedication as separate facets of contextual performance. , 1996 .

[10]  Jay Steffensmeier.  Situational constraints and personality as antecedents of organizational citizenship behaviors , 2008 .

[11]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[12]  Filip Lievens,et al.  A closer look at the frame-of-reference effect in personality scale scores and validity. , 2008, The Journal of applied psychology.

[13]  Gregory M. Hurtz,et al.  Personality and job performance: the Big Five revisited. , 2000, The Journal of applied psychology.

[14]  Newell K. Eaton,et al.  Criterion-related validities of personality constructs and the effect of response distortion on those validities , 1990 .

[15]  R. Piedmont,et al.  Predicting Supervisor Ratings of Job Performance Using the NEO Personality Inventory , 1994 .

[16]  Hannah R. Rothstein,et al.  Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. , 1990 .

[17]  Dan J. Putka,et al.  The Multifaceted Nature of Measurement Artifacts and Its Implications for Estimating Construct-Level Relationships , 2009 .

[18]  Kevin J. Eschleman,et al.  A meta-analytic examination of work and general locus of control. , 2010, The Journal of applied psychology.

[19]  Luke M. Mccormack,et al.  The Role of Personality in Leadership: An Application of the Five-Factor Model in the Australian Military , 2002 .

[20]  James M. Diefendorff,et al.  The Impact of Contextual Self‐Ratings and Observer Ratings of Personality on the Personality–Performance Relationship1 , 2006 .

[21]  Richard W Lowell,et al.  The Big Five and contextual performance: Expanding person-environment fit theory. , 2001 .

[22]  G. Mead,et al.  Mind, Self and Society. From the Standpoint of a Social Behaviorist. , 1935 .

[23]  Paula Caligiuri,et al.  THE BIG FIVE PERSONALITY CHARACTERISTICS AS PREDICTORS OF EXPATRIATE'S DESIRE TO TERMINATE THE ASSIGNMENT AND SUPERVISOR‐RATED PERFORMANCE , 2000 .

[24]  Erich C. Dierdorff,et al.  Dispositional Influences and Individual Differences in Work Performance: Relationships of Goal Orientation, Growth Need Strength, and Conscientiousness. , 2003 .

[25]  K. M. Kieffer,et al.  Person-Environment Congruence and Personality Domains in the Prediction of Job Performance and Work Quality. , 2004 .

[26]  M. Brannick,et al.  Improving Prediction of Work Performance Through Frame-of-Reference Consistency: Empirical Evidence Using Openness to Experience , 2010 .

[27]  V. Cook.  An Investigation of the Construct Validity of the Big Five Construct of Emotional Stability in Relation to Job Performance, Job Satisfaction, and Career Satisfaction , 2005 .

[28]  Michael J. Tews,et al.  Ability versus Personality , 2007 .

[29]  F. Morgeson,et al.  SELECTING INDIVIDUALS IN TEAM SETTINGS: THE IMPORTANCE OF SOCIAL SKILLS, PERSONALITY CHARACTERISTICS, AND TEAMWORK KNOWLEDGE , 2005 .

[30]  Theresa M. Welbourne,et al.  Managerial coping with organizational change: A dispositional perspective. , 1999 .

[31]  Gerard P. Hodgkinson,et al.  Development and validation of the Five-factor Model Questionnaire (FFMQ): An adjectival-based personality inventory for use in occupational settings , 2007 .

[32]  T. Judge,et al.  Loving yourself abundantly: relationship of the narcissistic personality to self- and other perceptions of workplace deviance, leadership, and task and contextual performance. , 2006, The Journal of applied psychology.

[33]  The customer orientation of financial advisers , 2008 .

[34]  Broad and Narrow Personality Traits in Relation to the Job Performance of Camp Counselors , 2005 .

[35]  In-Sue Oh,et al.  INCREMENTAL VALIDITY OF PERCEPTUAL SPEED AND ACCURACY OVER GENERAL MENTAL ABILITY , 2008 .

[36]  Murray R. Barrick,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS , 1991 .

[37]  R L Helmreich,et al.  Psychological predictors of astronaut effectiveness. , 1994, Aviation, space, and environmental medicine.

[38]  Marie Williams.  When is personality a predictor of performance? The moderating role of autonomy , 1999 .

[39]  T. Judge,et al.  INTERACTION AND INTERSECTION: THE CONSTELLATION OF EMOTIONAL STABILITY AND EXTRAVERSION IN PREDICTING PERFORMANCE , 2007 .

[40]  O. John,et al.  Big Five Inventory , 2012, Encyclopedia of Personality and Individual Differences.

[41]  M. Bing,et al.  Content Validity Does Matter for the Criterion-Related Validity of Personality Tests , 2009, Industrial and Organizational Psychology.

[42]  P. Costa,et al.  Revised NEO Personality Inventory (NEO-PI-R) and NEO-Five-Factor Inventory (NEO-FFI) , 1992 .

[43]  J. Salgado,et al.  Personality and Job Performance in Financial Services Managers , 1997 .

[44]  Murray R. Barrick,et al.  Personality and Performance at the Beginning of the New Millennium: What Do We Know and Where Do We Go Next? , 2001 .

[45]  Chockalingam Viswesvaran,et al.  Comparative analysis of the reliability of job performance ratings , 1996 .

[46]  J. M. Digman.  PERSONALITY STRUCTURE: EMERGENCE OF THE FIVE-FACTOR MODEL , 1990 .

[47]  Donald H. Kluemper,et al.  Evidence of Predictive and Incremental Validity of Personality Factors, Vocal Attractiveness and the Situational Interview , 2007 .

[48]  L. Hough.  The 'Big Five' Personality Variables--Construct Confusion: Description Versus Prediction , 1992 .

[49]  Nathan A. Bowling,et al.  A comparison of work-specific and general personality measures as predictors of work and non-work criteria , 2010 .

[50]  M. Bing,et al.  The predictive and interactive effects of equity sensitivity in teamwork‐oriented organizations , 2001 .

[51]  A. Ryan,et al.  Effects of Nonlinearity and Heteroscedasticity on the Validity of Conscientiousness in Predicting Overall Job Performance , 1999 .

[52]  M. Wesson,et al.  Psychological collectivism: a measurement validation and linkage to group member performance. , 2006, The Journal of applied psychology.

[53]  A. Minbashian,et al.  Complexity in the relationships among the subdimensions of extraversion and job performance in managerial occupations , 2009 .

[54]  M. Schmit,et al.  Frame-of-reference effects on personality scale scores and criterion-related validity. , 1995 .

[55]  James J. Connolly,et al.  Assessing the construct validity of a measure of learning agility , 2001 .

[56]  S. Rothmann,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE , 2003 .

[57]  Jeffrey M. Conte,et al.  Polychronicity, Big Five Personality Dimensions, and Sales Performance , 2005 .

[58]  John W. Lounsbury,et al.  The Development and Validation of a Personological Measure of Work Drive , 2003 .

[59]  Murray R. Barrick,et al.  Validity of observer ratings of the big five personality factors , 1994 .

[60]  D. Watson,et al.  Contextualized personality: traditional and new assessment procedures. , 2007, Journal of personality.

[61]  Michael P. Kirsch,et al.  METAANALYSES OF VALIDITY STUDIES PUBLISHED BETWEEN 1964 AND 1982 AND THE INVESTIGATION OF STUDY CHARACTERISTICS , 1984 .

[62]  J. Mowen,et al.  Understanding Compulsive Buying Among College Students: A Hierarchical Approach , 1999 .

[63]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[64]  Murray R. Barrick,et al.  Autonomy as a moderator of the relationships between the Big Five personality dimensions and job performance. , 1993 .

[65]  Neil Anderson,et al.  A meta-analytic study of general mental ability validity for different occupations in the European community. , 2003, The Journal of applied psychology.

[66]  F. M. Chan.  The Effects of Optimism and the Five-Factor Model of Personality on Stress and Performance in the Work Place , 2004 .

[67]  J. Salgado.  The Five Factor Model of personality and job performance in the European Community. , 1997, The Journal of applied psychology.

[68]  Thomas Sy,et al.  Relation of employee and manager emotional intelligence to job satisfaction and performance , 2006 .

[69]  L. A. Witt,et al.  Incremental Validity of Empirically Keyed Biodata Scales over GMA and the Five Factor Personality Constructs , 2000 .

[70]  Paul R. Sackett.  Final thoughts on the selection and assessment field , 2010 .

[71]  Beyond the Big Five: Increases in Understanding and Practical Utility , 2004 .

[72]  M. Tichon.  Personnel Selection in the Transportation Sector: An Investigation of Personality Traits in Relation to the Job Performance of Delivery Drivers , 2005 .

[73]  J. Diefendorff,et al.  EXAMINATION OF SITUATIONAL AND ATTITUDINAL MODERATORS OF THE HESITATION AND PERFORMANCE RELATION , 2006 .

[74]  E. Donahue,et al.  The Patterned Inconsistency of Traits: Mapping the Differential Effects of Social Roles on Self-Perceptions of the Big Five , 1998 .

[75]  James W. Westerman.  THE EFFECTS OF WORK ENVIRONMENT ON THE PERSONALITY-PERFORMANCE RELATIONSHIP: AN EXPLORATORY STUDY , 2007 .

[76]  John W. Jones.  Assessing privacy invasiveness of psychological test items: Job relevant versus clinical measures of integrity , 1991 .

[77]  B. Roberts,et al.  One personality, multiple selves: integrating personality and social roles. , 1994, Journal of personality.

[78]  G. Saucier.  Mini-markers: a brief version of Goldberg's unipolar big-five markers. , 1994, Journal of personality assessment.

[79]  Eric Sundstrom,et al.  Group Personality Composition and Performance in Military Service Teams , 2005 .

[80]  G. Norris.  Using measures of personality and self-efficacy to predict work performance / , 2002 .

[81]  Michael A. McDaniel,et al.  PUBLICATION BIAS: A CASE STUDY OF FOUR TEST VENDORS , 2006 .

[82]  T. Judge,et al.  IN SUPPORT OF PERSONALITY ASSESSMENT IN ORGANIZATIONAL SETTINGS , 2007 .

[83]  Murray R. Barrick,et al.  Personality and job performance: test of the mediating effects of motivation among sales representatives. , 2002, The Journal of applied psychology.

[84]  W. Mischel.  Toward a cognitive social learning reconceptualization of personality. , 1973, Psychological review.

[85]  R. Morrison.  Mind, Self and Society from the Standpoint of a Social Behaviorist , 1936 .

[86]  A. Ryan,et al.  Effects of Item Context Specificity on the Measurement Equivalence of a Personality Inventory , 2000 .

[87]  Matthew S. O’Connell,et al.  Incremental Validity of Locus of Control After Controlling for Cognitive Ability and Conscientiousness , 2005 .

[88]  Jordan B Peterson,et al.  Prefrontal cognitive ability, intelligence, Big Five personality, and the prediction of advanced academic and workplace performance. , 2007, Journal of personality and social psychology.

[89]  Daniel Heller,et al.  The influence of work personality on job satisfaction: incremental validity and mediation effects. , 2009, Journal of personality.

[90]  M. Born,et al.  Personal and Situational Determinants of Personality Responses: A Partial Reanalysis and Reinterpretation of the Schmit et al. (1995) Data , 2001 .

[91]  L. Witt,et al.  Moderators of the openness to experience‐performance relationship , 2002 .

[92]  B. Hesketh,et al.  Why Openness to Experience is not a Good Predictor of Job Performance , 2004 .

[93]  Murray R. Barrick,et al.  The Joint Relationship of Conscientiousness and Ability with Performance: Test of the Interaction Hypothesis , 1999 .

[94]  S. Rynes,et al.  Applicant reactions to alternative selection procedures , 1993 .

[95]  Tom J. Brown,et al.  The Customer Orientation of Service Workers: Personality Trait Effects on Self-and Supervisor Performance Ratings , 2002 .

[96]  G. Stewart,et al.  Personality dimensions and domains of service performance: A field investigation , 1995 .

[97]  B. Roberts,et al.  Cross-sectional and longitudinal tests of the Personality and Role Identity Structural Model (PRISM). , 2006, Journal of personality.

[98]  R. Guion,et al.  Validity Of Personality Measures In Personnel Selection , 1965 .

[99]  Chet Robie,et al.  DEVELOPMENT OF A GLOBAL MEASURE OF PERSONALITY , 2000 .

[100]  Gregory Charles Cutchin.  Relationships between the big five personality factors and performance criteria for inservice high school teachers , 1998 .

[101]  Dawn S. Carlson,et al.  The work-family interface and job performance: moderating effects of conscientiousness and perceived organizational support. , 2006, Journal of occupational health psychology.

[102]  J. Hunter.  Cognitive ability, cognitive aptitudes, job knowledge, and job performance , 1986 .

[103]  Characteristics of Successful Direct Support Professionals: An Examination of Personality and Cognitive Ability Requirements , 2010 .

[104]  Rick R. Jacobs,et al.  Selecting Bus Drivers: Multiple Predictors, Multiple Perspectives on Validity, and Multiple Estimates of Utility , 1996 .

[105]  Murray R. Barrick,et al.  ANTECEDENTS OF INVOLUNTARY TURNOVER DUE TO A REDUCTION IN FORCE , 1994 .

[106]  Kwanghyun Kim.  An investigation of the relationship between cultural competence and expatriate job performance , 2008 .

[107]  M. Bing,et al.  Incremental validity of the frame-of-reference effect in personality scale scores: a replication and extension. , 2004, The Journal of applied psychology.

[108]  Robert E. Ployhart,et al.  Testing the Rules of Justice: The Effects of Frame-of-Reference and Pre-Test Validity Information on Personality Test Responses and Test Perceptions , 2005 .

[109]  W. Mischel,et al.  A conditional approach to dispositional constructs: the local predictability of social behavior. , 1987, Journal of personality and social psychology.

[110]  Berrin Erdogan,et al.  A longitudinal study of the moderating role of extraversion: leader-member exchange, performance, and turnover during new executive development. , 2006, The Journal of applied psychology.

[111]  Robert G. Lord,et al.  Examining the roles of job involvement and work centrality in predicting organizational citizenship behaviors and job performance , 2002 .

[112]  R. L. Dipboye,et al.  RECONSIDERING THE USE OF PERSONALITY TESTS IN PERSONNEL SELECTION CONTEXTS , 2007 .

[113]  Zhang Ying-hong.  On Work Analysis , 2008 .

[114]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[115]  Gordon W. Allport,et al.  Pattern and growth in personality , 1961 .

[116]  L. R. Goldberg.  THE DEVELOPMENT OF MARKERS FOR THE BIG-FIVE FACTOR STRUCTURE , 1992 .

[117]  A. Neubauer,et al.  Broad and narrow personality traits for predicting managerial success , 2010 .

[118]  Validity Study Relationship between the Performance Perspectives Inventory's Implementing Scale and Job Performance of Incumbents in a Dual Security/Service Role , 2003 .

[119]  P. Sackett,et al.  Effects of Self-Deceptive Enhancement on Personality-Job Performance Relationships , 2007 .

[120]  Leslie B. Hammer,et al.  A field study of frame-of-reference effects on personality test validity. , 2003, The Journal of applied psychology.

[121]  Thomas A. Timmerman.  Relationships Between NEO PI-R Personality Measures And Job Performance Ratings of Inbound Call Center Employees , 2004 .

[122]  N. Schmitt,et al.  Situational Judgment and Job Performance , 2002 .

[123]  Eyran Kraus.  Personality and job performance: The mediating roles of leader-member exchange quality and action control. , 2002 .

[124]  Craig M. Reddock,et al.  The Relationship of Reliability and Validity of Personality Tests to Frame-of-Reference Instructions and Within-Person Inconsistency , 2011 .

[125]  J. Rosse,et al.  A field study of job applicants' reactions to personality and cognitive ability testing. , 1994 .

[126]  W. Mischel,et al.  A conditional approach to dispositional constructs: the local predictability of social behavior. , 1987, Journal of personality and social psychology.

[127]  L. Hough,et al.  Personality : Its Measurement and Validity for Employee Selection , 2010 .

[128]  Edwin E. Ghiselli,et al.  THE VALIDITY OF APTITUDE TESTS IN PERSONNEL SELECTION , 1973 .