Is Justice the Same for Everyone? Examining Fairness Items Using Multiple-group Analysis

PurposeThe purpose of this study was to examine whether fairness assessed in a widely used multisource instrument written by practitioners possessed a similar factor structure as fairness measured in academic literature, and whether different groups based on their relationship to the ratee (e.g., peers, subordinates, supervisors) perceived a similar structure to the construct.Design/methodology/approachMultisource data were collected for 141 managers from a variety of organizations across the United States, who participated in a leadership development program offered by Personnel Decisions International (PDI), a global management consulting firm.FindingsThe study results show that fairness measured with a widely used multisource instrument indeed assesses the same construct as that examined in the academic literature. Peers, subordinates, and supervisors perceived fairness similarly, whereas self-reports of fairness differed, suggesting that they function as a systematic source of variance in the measurement of fairness.ImplicationsThe findings in this study demonstrate that leaders can be confident that their constituents are all conceptualizing fairness the same way, such that differences are meaningful and not due to differences in the understanding of the items in the instrument.Originality/valueThe proliferation of the use of multisource feedback instruments in leadership development programs has facilitated the assessment of the fairness of managers. This study shows that perceptions of fairness of the manager as reported by his or her constituencies (e.g., peers, supervisor, subordinates) change very little as a function of who is doing the perceiving, which heretofore has not been shown.

[1]  T. Tyler,et al.  Understanding why the justice of group procedures matters: A test of the psychological dynamics of the group-value model. , 1996 .

[2]  M. Audrey Korsgaard,et al.  Procedural Justice in Performance Evaluation: The Role of Instrumental and Non-Instrumental Voice in Performance Appraisal Discussions , 1995 .

[3]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[4]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[5]  Gary P. Latham,et al.  LEADERSHIP TRAINING IN ORGANIZATIONAL JUSTICE TO INCREASE CITIZENSHIP BEHAVIOR WITHIN A LABOR UNION: A REPLICATION , 1997 .

[6]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[7]  W. Bennis,et al.  Don't hire the wrong CEO. , 2000, Harvard business review.

[8]  Maureen L. Ambrose,et al.  Are Procedural Justice and Distributive Justice Conceptually Distinct , 2005 .

[9]  M. Deutsch Equity, Equality, and Need: What Determines Which Value Will Be Used as the Basis of Distributive Justice? , 1975 .

[10]  F. Floyd,et al.  Factor analysis in the development and refinement of clinical assessment instruments. , 1995 .

[11]  Gordon W. Cheung,et al.  Testing Factorial Invariance across Groups: A Reconceptualization and Proposed New Method , 1999 .

[12]  Stuart S. Nagel,et al.  Procedural Justice: A Psychological Analysis , 1976 .

[13]  E. Lind,et al.  The injustices of others: Social reports and the integration of others' experiences in organizational justice judgments. , 2002 .

[14]  Nancy A. Welsh,et al.  Perceptions of Fairness , 2006 .

[15]  Robert A. Giacalone,et al.  Applied Impression Management: How Image-Making Affects Managerial Decisions , 1991 .

[16]  Peter Y. Chen,et al.  Using Social Exchange Theory to Distinguish Procedural from Interactional Justice , 2002 .

[17]  Maureen L. Ambrose,et al.  Procedural and distributive justice are more similar than you think: A monistic perspective and a research agenda. , 2001 .

[18]  Gordon W. Cheung Multifaceted conceptions of self–other ratings disagreement. , 1999 .

[19]  Joyce E. Bono,et al.  UNDERSTANDING RESPONSES TO MULTI‐SOURCE FEEDBACK: THE ROLE OF CORE SELF‐EVALUATIONS , 2005 .

[20]  A. Panter,et al.  Writing about structural equation models. , 1995 .

[21]  G. Leventhal,et al.  The Distribution of Rewards and Resources in Groups and Organizations , 1976 .

[22]  Jerald Greenberg,et al.  How should organizational justice be measured , 2005 .

[23]  T. Tyler,et al.  The influence of outcomes and procedures on satisfaction with formal leaders. , 1981 .

[24]  R. Bies,et al.  Beyond distrust: "Getting even" and the need for revenge. , 1996 .

[25]  B. Thompson,et al.  Factor Analytic Evidence for the Construct Validity of Scores: A Historical Overview and Some Guidelines , 1996 .

[26]  R. Bies,et al.  Establishing fairness in the eye of the beholder: Managing impressions of organizational justice. , 1991 .

[27]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[28]  J. Colquitt On the dimensionality of organizational justice: a construct validation of a measure. , 2001, The Journal of applied psychology.

[29]  Mark R. Edwards,et al.  360゜ feedback : the powerful new model for employee assessment & performance improvement , 1996 .

[30]  Stéphane Brutus,et al.  Multisource assessment programs in organizations: An insider's perspective , 2002 .

[31]  Leanne E. Atwater,et al.  360-Degree Feedback to Leaders , 2006 .

[32]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[33]  William A. Gentry,et al.  Competencies for Leadership Development: What's Hot and What's Not When Assessing Leadership-Implications for Organization Development , 2007 .

[34]  Robert H. Moorman,et al.  Relationship between organizational justice and organizational citizenship behaviors : do fairness perceptions influence employee citizenship ? , 1991 .

[35]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[36]  Kyle Lewis,et al.  Integrating Justice and Social Exchange: The Differing Effects of Fair Procedures and Treatment on Work Relationships , 2000 .

[37]  R. Lewicki Research on Negotiation in Organizations , 1990 .

[38]  J. S. Adams,et al.  Inequity In Social Exchange , 1965 .

[39]  Russell Cropanzano,et al.  Justice in the workplace: Approaching fairness in human resource management. , 1993 .

[40]  R. Folger,et al.  Effects of Procedural and Distributive Justice on Reactions to Pay Raise Decisions , 1989 .

[41]  Jerald Greenberg,et al.  Handbook of Organizational Justice , 2005 .

[42]  Nambury S. Raju,et al.  Peer and subordinate performance appraisal measurement equivalence. , 1998 .

[43]  Jerald Greenberg,et al.  Advances in Organizational Justice , 2001 .

[44]  Rajnandini Pillai,et al.  ARE THE SCALES TIPPED IN FAVOR OF PROCEDURAL OR DISTRIBUTIVE JUSTICE? AN INVESTIGATION OF THE U.S., INDIA, GERMANY, AND HONG KONG (CHINA) , 2001 .

[45]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[46]  Jerald Greenberg,et al.  Cultivating an Image of Justice: Looking Fair on the Job , 1988 .

[47]  Paul E. Spector,et al.  The Role of Justice in Organizations: A Meta-Analysis , 2001 .

[48]  Robert M. Guion,et al.  Assessment, Measurement, and Prediction for Personnel Decisions , 1997 .

[49]  Lynne Andersson,et al.  When Workers Flout Convention: A Study of Workplace Incivility , 2001 .

[50]  R. Cattell The Scree Test For The Number Of Factors. , 1966, Multivariate behavioral research.

[51]  J. Greenberg,et al.  Losing sleep over organizational injustice: attenuating insomniac reactions to underpayment inequity with supervisory training in interactional justice. , 2006, The Journal of applied psychology.

[52]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[53]  T. Judge,et al.  Organizational justice and stress: the mediating role of work-family conflict. , 2004, The Journal of applied psychology.

[54]  David W. Bracken,et al.  The handbook of multisource feedback : the comprehensive resource for designing and implementing MSF processes , 2001 .

[55]  W. Cooley,et al.  Multivariate Data Analysis. , 1973 .

[56]  J. Greenberg,et al.  The social side of fairness: Interpersonal and informational classes of organizational justice. , 1993 .

[57]  James W. Smither,et al.  DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL, META‐ANALYSIS, AND REVIEW OF EMPIRICAL FINDINGS , 2005 .

[58]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[59]  Allen I. Huffcutt,et al.  Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. , 1997 .

[60]  Donald E. Conlon,et al.  Justice at the millennium: a meta-analytic review of 25 years of organizational justice research. , 2001, The Journal of applied psychology.

[61]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[62]  John Schaubroeck,et al.  A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. , 1988 .

[63]  David Antonioni,et al.  Designing an effective 360-degree appraisal feedback process , 1996 .

[64]  R. Hoyle Structural equation modeling: concepts, issues, and applications , 1997 .

[65]  Walter C. Borman,et al.  The rating of individuals in organizations: An alternate approach , 1974 .

[66]  T. Little Mean and Covariance Structures (MACS) Analyses of Cross-Cultural Data: Practical and Theoretical Issues. , 1997, Multivariate behavioral research.

[67]  Paul D. Sweeney,et al.  Process and outcome: gender differences in the assessment of justice , 1997 .

[68]  R. Kanfer,et al.  Equivalence of psychological measurement in heterogeneous populations. , 1985, The Journal of applied psychology.

[69]  H. Kaiser An index of factorial simplicity , 1974 .

[70]  L. Gómez-Mejia,et al.  GAINSHARING AND MUTUAL MONITORING: A COMBINED AGENCY-ORGANIZATIONAL JUSTICE INTERPRETATION , 1995 .

[71]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[72]  Winston Bennett,et al.  Assessing measurement equivalence across rating sources: a multitrait-multirater approach. , 2005, The Journal of applied psychology.

[73]  D. Messick,et al.  Why we are fairer than others: A cross-cultural replication and extension , 1986 .

[74]  Walter C. Borman,et al.  History and development of multisource feedback as a methodology , 2001 .

[75]  Leanne E. Atwater The Power of 360° Feedback: How to Leverage Performance Evaluations for Top Productivity , 1998 .

[76]  Fred Dansereau,et al.  Leadership and Empowerment: A Social Exchange Perspective , 1995 .

[77]  Chet Robie,et al.  Self-Other Agreement: Comparing its Relationship with Performance in the U.S. and Europe , 2005 .

[78]  Gary P. Latham,et al.  Increasing citizenship behavior within a labor union : A test of organizational justice theory , 1996 .

[79]  R. Folger Distributive and procedural justice: Combined impact of voice and improvement on experienced inequity. , 1977 .

[80]  Paul Kline,et al.  An easy guide to factor analysis , 1993 .

[81]  James W. Smither,et al.  CAN MULTI-SOURCE FEEDBACK CHANGE PERCEPTIONS OF GOAL ACCOMPLISHMENT, SELF-EVALUATIONS, AND PERFORMANCE-RELATED OUTCOMES? THEORY-BASED APPLICATIONS AND DIRECTIONS FOR RESEARCH , 1995 .

[82]  T. Tyler,et al.  Exploring the function of legitimacy: Mediating effects of personal and institutional legitimacy on leadership endorsement and system support. , 1985 .