Transsituational variability in human performance within assessment centers

Abstract Current models in the psychology of personality describe behavior as largely a function of the situation (i.e., situation specific) rather than reflective of broad underlying stable traits (i.e., consistent). Because of sample size limitations and inappropriate levels of abstraction in performance measures, this “interactionist” explanation of behavior has rarely been formally evaluated in industrial/organizational psychology. In this study 2056 assessment center candidates were rated on eight skills or abilities (traits) measured over five situational exercises (situations). Person, situation, and trait variance components were identified via the multitrait—multimethod matrix analysis. The results revealed major sources of variance from person and person × situation components, high convergent validity in the ratings, but a lack of discriminant validity across specialized skills and abilities. Assessees performed differently in the various situational exercises (i.e., person × situation interaction), although differential ratings were probably due largely to halo error. The findings are discussed in the context of substantive and methodological issues relating to transsituational variability in behavior.

[1]  J. B. Maller,et al.  Studies in the Nature of Character. , 1930 .

[2]  R. Moos.  Sources of variance in responses to questionnaires and in behavior. , 1969, Journal of abnormal psychology.

[3]  Paul Slovic,et al.  Comparison of Bayesian and Regression Approaches to the Study of Information Processing in Judgment. , 1971 .

[4]  C. L. Jaffee,et al.  BLACKS SUPERVISING WHITES: A STUDY OF INTERRACIAL DIFFICULTIES IN WORKING TOGETHER IN A SIMULATED ORGANIZATION1 , 1972 .

[5]  Karl O. Magnusen.  Organizational Design, Development, and Behavior: A Situational View , 1977 .

[6]  H. Kelley.  Attribution theory in social psychology , 1967 .

[7]  D. W. Fiske.  The Constraints on Intra-Individual Variability in Test Responses1 , 1957 .

[8]  D. Magnusson,et al.  Toward an interactional psychology of personality. , 1976, Psychological bulletin.

[9]  E. K. Taylor,et al.  THE MISSING MIDDLE IN VALIDATION RESEARCH , 1976 .

[10]  John E. Hunter,et al.  Moderator Research and the Law of Small Numbers. , 1978 .

[11]  D. Bray,et al.  The assessment center in the measurement of potential for business management. , 1966, Psychological monographs.

[12]  John E. Hunter,et al.  Statistical power in criterion-related validation studies. , 1976 .

[13]  Julian C. Stanley,et al.  Analysis of unreplicated three-way classifications, with applications to rater bias and trait independence , 1961 .

[14]  R. Moos.  Situational analysis of a therapeutic community milieu. , 1968, Journal of abnormal psychology.

[15]  P. Wachtel.  Psychodynamics, behavior therapy, and the implacable experimenter: an inquiry into the consistency of personality. , 1973, Journal of abnormal psychology.

[16]  S. Zedeck,et al.  Nursing performance as measured by behavioral expectation scales: A multitrait-multirater analysis , 1972 .

[17]  Joseph L. Moses,et al.  Relationship of assessment-center performance to management progress of women. , 1975 .

[18]  D. Bem.  Constructing cross-situational consistencies in behavior: some thoughts on Alker's critique of Mischel. , 1972, Journal of personality.

[19]  W. C. Byham.  The Assessment Center As An Aid In Management Development. , 1971 .

[20]  John R. Hinrichs,et al.  An Eight-Year Follow-Up of a Management Assessment Center. , 1978 .

[21]  K S Bowers,et al.  Situationism in psychology: an analysis and a critique. , 1973, Psychological review.

[22]  J. Hinrichs,et al.  Reliability of Measurement in Situational Exercises: An Assessment of the Assessment Center Method. , 1976 .

[23]  R. Harré,et al.  The explanation of social behaviour , 1973 .

[24]  R. Abelson.  Sex Differences in Predictability of College Grades , 1951 .

[25]  Edwin E. Ghiselli,et al.  Differentiation of Tests in Terms of the Accuracy with which they Predict for a Given Individual , 1960 .

[26]  S R WALLACE,et al.  CRITERIA FOR WHAT? , 1965, The American psychologist.

[27]  W. S. Jones,et al.  Assessing corporate talent;: A key to managerial manpower planning, , 1970 .

[28]  N. Endler,et al.  The Role of Person-by-Situation Interactions in Personality Theory , 1977 .

[29]  M. Wallach,et al.  Testing the hypothesis that a person will be consistent: stylistic consistency versus situational specificity in size of children's drawings. , 1972, Journal of personality.

[30]  R. Cattell.  Fluctuation of Sentiments and Attitudes as a Measure of Character Integration and of Temperament , 1943 .

[31]  Douglas W. Bray,et al.  Management Assessment Center Evaluations and Subsequent Job Performance of White and Black Females. , 1976 .

[32]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[33]  J. Hinrichs.  Comparison of "real life" assessments of management potential with situational exercises, paper-and-pencil ability tests, and personality inventories. , 1969, The Journal of applied psychology.

[34]  R. Klimoski,et al.  ASSESSMENT CENTERS—VALID OR MERELY PRESCIENT , 1977 .

[35]  W. T. Norman,et al.  Adjustment and academic predictability. , 1954 .

[36]  N. Endler,et al.  The person versus the situation--a pseudo issue? A response to Alker. , 1973, Journal of personality.

[37]  G. Āllport.  Personality: A Psychological Interpretation , 1938 .

[38]  B. Ekehammar,et al.  Interactionism in personality from a historical perspective. , 1974, Psychological bulletin.

[39]  R. Berdie,et al.  Intra-Individual Variability and Predictability , 1961 .

[40]  D W FISKE,et al.  Intra-individual response variability. , 1955, Psychological bulletin.

[41]  W. J. Mcnamara,et al.  Interrater reliability in situational tests. , 1967, The Journal of applied psychology.

[42]  L. G. Rorer,et al.  Illusory correlation and subjective judgment. , 1972 .

[43]  W. Mischel.  Toward a cognitive social learning reconceptualization of personality. , 1973, Psychological review.

[44]  W. Holtzman,et al.  The Survey of Study Habits and Attitudes: a New Instrument for the Prediction of Academic Success , 1954 .

[45]  N. Endler,et al.  The case for person-situation interactions. , 1975 .

[46]  R. J. Campbell,et al.  Formative years in business: A long-term AT&T study of managerial lives , 1974 .

[47]  C. A. Vale,et al.  Individual differences and general laws in psychology: A reconciliation. , 1969 .

[48]  Michael J. Kavanagh,et al.  Issues in managerial performance: Multitrait-multimethod analyses of ratings. , 1971 .

[49]  F. Fiedler.  Validation and extension of the contigency model of leadership effectiveness: a review of empirical findings. , 1971, Psychological bulletin.

[50]  E. E. Jones,et al.  The actor and the observer: Divergent perceptions of the causes of behavior. , 1972 .

[51]  N. Campus.  Transituational consistency as a dimension of personality. , 1974, Journal of personality and social psychology.

[52]  M. Hakel,et al.  Temporal stability and individual differences in using assessment information to form overall ratings , 1979 .

[53]  E. Ghiselli.  Differentiation of individuals in terms of their predictability. , 1956 .

[54]  James O. Mitchel.  Assessment center validity: A longitudinal study. , 1975 .

[55]  D. R. Saunders.  Moderator Variables in Prediction , 1956 .

[56]  N. Endler,et al.  Estimating Variance Components from Mean Squares for Random and Mixed Effects Analysis of Variance Models , 1966 .

[57]  N. Endler,et al.  S-R inventories of hostility and comparisons of the proportions of variance from persons, responses, and situations for hostility and anxiousness. , 1968, Journal of personality and social psychology.

[58]  R. Boruch,et al.  Alternative Methods of Analysis: Multitrait-Multimethod Data , 1970 .

[59]  Walter C. Borman,et al.  Exploring upper limits of reliability and validity in job performance ratings. , 1978 .

[60]  Edwin E. Ghiselli,et al.  Moderating effects and differential reliability and validity. , 1963 .

[61]  Robert L. Holzbach,et al.  Rater bias in performance ratings: Superior, self-, and peer ratings. , 1978 .

[62]  T. Dickinson,et al.  A multitrait-multi method analysis of scales developed by retranslation , 1973 .

[63]  Norman S. Endler,et al.  Interactional psychology and personality , 1976 .

[64]  J. Cummings.  VARIABILITY OF JUDGEMENT AND ‘STEADINESS’ OF CHARACTER , 1939 .

[65]  Edwin E. Ghiselli,et al.  The Prediction of Predictability , 1960 .

[66]  W. L. Layton.  The Variability of Individuals Scores Upon Successive Testings on the Minnesota Multiphasic Personality Inventory , 1954 .

[67]  W. Mischel,et al.  Continuity and change in personality. , 1969, The American psychologist.

[68]  S. R. Wallace.  HOW HIGH THE VALIDITY?1 , 1974 .

[69]  H. A. Thomson.  Comparison of predictor and criterion judgments of managerial performance using the multitrait-multimethod approach. , 1970 .

[70]  N. Schmitt.  Interrater agreement in dimensionality and combination of assessment center judgments. , 1977 .

[71]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[72]  N. Schmitt,et al.  Sex and race composition of assessment center groups as a determinant of peer and assessor ratings , 1977 .

[73]  N. Endler,et al.  Generalizability of contributions from sources of variance in the S-R inventories of anxiousness. , 1969, Journal of personality.

[74]  C. L. Jaffee,et al.  Supervisory Selection Program for Disadvantaged or Minority Employees. , 1972 .

[75]  H. Alker.  Is personality situationally specific or intrapsychically consistent , 1972 .

[76]  E. Lawler,et al.  The multitrait-multirater approach to measuring managerial job performance. , 1967, The Journal of applied psychology.

[77]  Norman Frederiksen,et al.  Differential Predictability in the Use of Test Scores , 1953 .

[78]  Daryl J. Bem,et al.  ON PREDICTING SOME OF THE PEOPLE SOME OF THE TIME: THE SEARCH FOR CROSS-SITUATIONAL CONSISTENCIES IN BEHAVIOR , 1974 .

[79]  Benjamin Schneider,et al.  PERSON-SITUATION SELECTION: A REVIEW OF SOME ABILITY-SITUATION INTERACTION RESEARCH , 1978 .

[80]  T. Newcomb.  The consistency of certain extrovert-introvert behavior patterns in 51 problem boys. , 1929 .

[81]  R. Stagner.  The Relation of Personality to Academic Aptitude and Achievement , 1933 .