The Cross-Cultural Equivalence of Job Performance Ratings

The primary purpose of this study was to assess the cross-cultural invariance of job performance ratings. A secondary purpose was to examine potential cross-cultural differences in correlates of performance ratings (i.e., ratee sex, age, tenure; supervisor's opportunity to observe ratee). Fast-food supervisors from Canada, South Korea, and Spain rated employees on their technical proficiency, customer service, and teamwork. Results show that these ratings demonstrate a basic level of measurement invariance, although the error variances of the ratings and pattern of construct variances and covariances were largely culture-specific. This suggests that supervisors across cultures may use and interpret the ratings similarly, but perceive differences in performance. Furthermore, age, tenure, and the supervisor's opportunity to observe the ratee were found to affect ratings differently across cultures. Overall, this study suggests that although job performance ratings are at least partially invariant across cultures, latent performance may not be, and we present some preliminary data as to why latent invariance may not exist.

[1]  James T. Austin,et al.  From then to now: The development of industrial-organizational psychology in the United States. , 1992 .

[2]  H. Triandis,et al.  Etic Plus Emic Versus Pseudoetic , 1983 .

[3]  Ann Marie Ryan,et al.  AN INTERNATIONAL LOOK AT SELECTION PRACTICES: NATION AND CULTURE AS EXPLANATIONS FOR VARIABILITY IN PRACTICE , 1999 .

[4]  H. Triandis Cross-cultural industrial and organizational psychology. , 1994 .

[5]  William P. Anthony,et al.  Human Resource Management: A Strategic Approach , 2001 .

[6]  A. Parasuraman,et al.  More on improving service quality measurement , 1993 .

[7]  J. H. Steiger Structural Model Evaluation and Modification: An Interval Estimation Approach. , 1990, Multivariate behavioral research.

[8]  Nambury S. Raju,et al.  Peer and subordinate performance appraisal measurement equivalence. , 1998 .

[9]  Randall S. Schuler,et al.  Understanding Compensation Practice Variations Across Firms: The Impact of National Culture , 1998 .

[10]  Peter J. Dowling,et al.  An Integrative Framework of Strategic International Human Resource Management , 1993 .

[11]  Walter C. Borman,et al.  Job behavior, performance, and effectiveness. , 1991 .

[12]  Wayne F. Cascio,et al.  Cumulative evidence of the relationship between employee age and job performance. , 1989 .

[13]  Walter C. Borman,et al.  Task Performance and Contextual Performance: The Meaning for Personnel Selection Research , 1997 .

[14]  N. Schmitt,et al.  Models of Job Performance Ratings: An Examination of Ratee Race, Ratee Gender, and Rater Level Effects , 1996 .

[15]  Howard Thomas,et al.  Global strategic analyses: Frameworks and approaches , 1999 .

[16]  Olivier Furrer,et al.  The Relationships between Culture and Service Quality Perceptions , 2000 .

[17]  D. Chan Tightness-Looseness Revisited: Some Preliminary Analyses in Japan and the United States , 1996 .

[18]  U. Kim,et al.  Indigenous analysis of collective representations: A Korean perspective. , 1993 .

[19]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[20]  Y. Aharoni Cultures and Organizations: Software of the Mind , 1992 .

[21]  Alice N. Outerbridge,et al.  Impact of job experience and ability on job knowledge, work sample performance, and supervisory ratings of job performance , 1986 .

[22]  P. J. Pelto,et al.  The differences between “tight” and “loose” societies , 1968 .

[23]  Chris Brewster,et al.  Towards a ‘European’ Model of Human Resource Management , 1995 .

[24]  Barbara W. Keats,et al.  Navigating in the new competitive landscape: Building strategic flexibility and competitive advantage in the 21st century , 1998 .

[25]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[26]  S. Schwartz Beyond individualism/collectivism: New cultural dimensions of values. , 1994 .

[27]  W. Borman,et al.  Expanding the Criterion Domain to Include Elements of Contextual Performance , 1993 .

[28]  Chet Robie,et al.  DEVELOPMENT OF A GLOBAL MEASURE OF PERSONALITY , 2000 .

[29]  C. Werts,et al.  The Use Of Analysis Of Covariance Structures For Comparing The Psychometric Properties Of Multiple Variables Across Populations. , 1978, Multivariate behavioral research.

[30]  A. Goldberger,et al.  Estimation of a Model with Multiple Indicators and Multiple Causes of a Single Latent Variable , 1975 .

[31]  Susan S. White,et al.  Linking service climate and customer perceptions of service quality: test of a causal model. , 1998, The Journal of applied psychology.

[32]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[33]  A. Mattila The Role of Culture in the Service Evaluation Process , 1999 .

[34]  Peter J. Dowling,et al.  Strategic performance measurement and management in multinational corporations , 1991 .

[35]  G. Hofstede,et al.  Culture′s Consequences: International Differences in Work-Related Values , 1980 .

[36]  Deborah L. Whetzel,et al.  Applied measurement methods in industrial psychology , 1997 .

[37]  John E. Hunter,et al.  Job experience correlates of job performance. , 1988 .

[38]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[39]  K. Murphy Personnel Selection in Organizations , 1992 .

[40]  Paul Sparrow,et al.  Redefining the field of European human resource management: A battle between national mindsets and forces of business transition? , 1997 .

[41]  Bengt Muthen,et al.  Some uses of structural equation modeling in validity studies: Extending IRT to external variables , 1986 .

[42]  Bruce J. Avolio,et al.  Age and Work Performance in Nonmanagerial Jobs: The Effects of Experience and Occupational Type , 1990 .

[43]  Bruce J. Avolio,et al.  A meta-analysis of age differences in job performance. , 1986 .

[44]  Joo-yup Kim,et al.  NATIONAL CULTURE AND THE FACTORS AFFECTING PERCEPTIONS OF PAY FAIRNESS IN KOREA AND THE UNITED STATES , 1997 .

[45]  Hannah R. Rothstein,et al.  Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. , 1990 .

[46]  Robert J. Gerard,et al.  Winning the Service Game , 1995 .

[47]  Tony Grundy,et al.  Human resource management—a strategic approach , 1997 .

[48]  An Extension of the Propensity Score Adjustment Method for the Analysis of Group Differences in MIMIC Models. , 1999, Multivariate behavioral research.

[49]  A. Parasuraman,et al.  Refinement and reassessment of the SERVQUAL scale. , 1991 .

[50]  M. Hitt,et al.  PRIVATIZATION AND ENTREPRENEURIAL TRANSrORMATION: EMERGING ISSUES AND A rUTURE RESEARCH AGENDA , 2000 .

[51]  R. Schuler,et al.  Managing worldwide diversity at Pepsi-Cola International. , 1992 .

[52]  Peter Villanova,et al.  The criterion problem: 1917–1992. , 1992 .

[53]  R. Kanfer,et al.  Equivalence of psychological measurement in heterogeneous populations. , 1985, The Journal of applied psychology.

[54]  Walter C. Borman,et al.  Effects of ratee task performance and interpersonal factors on supervisor and peer performance ratings , 1995 .

[55]  Walter C. Borman,et al.  Examination of race and sex effects on performance ratings , 1989 .

[56]  H. Triandis,et al.  Multistrategy Approach to Cross-Cultural Research , 1983 .

[57]  Oded Shenkar Global perspectives of human resource management , 1995 .

[58]  David Chan,et al.  EMPLOYEE ATTITUDE SURVEYS IN A MULTINATIONAL ORGANIZATION: CONSIDERING LANGUAGE AND CULTURE IN ASSESSING MEASUREMENT EQUIVALENCE , 1999 .

[59]  Gerald R. Ferris,et al.  Social Context of Performance Evaluation Decisions , 1993 .

[60]  P. Christopher Earley,et al.  Impetus for action: A cultural analysis of justice and , 1997 .

[61]  G. Hofstede,et al.  Cultures and Organizations: Software of the Mind , 1991 .

[62]  C. Brewster,et al.  Human Resource Management in Northern Europe: Trends, Dilemmas and Strategy , 2000 .

[63]  Charles L. Hulin,et al.  A Psychometric Theory of Evaluations of Item and Scale Translations , 1987 .

[64]  S. J. Goff,et al.  Joint relation of experience and ability with job performance: Test of three hypotheses. , 1988 .

[65]  Kenneth S. Law,et al.  Organizational citizenship behavior: Comparing perspectives of supervisors and subordinates across four international samples , 1999 .

[66]  C. Brewster,et al.  Human Resource Management in Northern Europe , 2000 .