Validating Surrogate Measures of Psychological Constructs: The Application of Construct Equivalence to Archival Data

Archival data present opportunities to test important research questions. Unfortunately, archival data do not always include validated measures of the psychological constructs that may be of interest to researchers. In this article, the authors describe how the construct equivalence approach can be applied to organizational archival data to derive and empirically validate surrogate measures of psychological constructs. A step-by-step explanation of how to validate such measures, which includes the use of confirmatory factor analysis to examine the data from a multitrait-multimethod perspective, is described. This approach facilitates the use of organizational archives for research purposes.

[1]  K. Bollen.  A New Incremental Fit Index for General Structural Equation Models , 1989 .

[2]  Rick H. Hoyle,et al.  Structural models for multimode designs in personality and temperament research. , 1994 .

[3]  J. H. Steiger.  Structural Model Evaluation and Modification: An Interval Estimation Approach. , 1990, Multivariate behavioral research.

[4]  James L. Arbuckle,et al.  Amos User''s Guide , 1995 .

[5]  Newell K. Eaton,et al.  Criterion-related validities of personality constructs and the effect of response distortion on those validities , 1990 .

[6]  C. Halverson,et al.  The Developing structure of temperament and personality from infancy to adulthood , 1994 .

[7]  D. Campbell,et al.  Unobtrusive Measures: Nonreactive Research in the Social Sciences , 1966 .

[8]  Peter M. Bentler,et al.  Practical Issues in Structural Modeling , 1987 .

[9]  John P. Meyer,et al.  The measurement and antecedents of affective, continuance and normative commitment to the organization , 1990 .

[10]  J. P. Wanous,et al.  Single-Item Reliability: A Replication and Extension , 2001 .

[11]  J. Mathieu,et al.  A review and meta-analysis of the antecedents, correlates, and consequences of organizational commitment , 1990 .

[12]  Herbert W. Marsh,et al.  Latent variable models of multitrait-multimethod data. , 1995 .

[13]  G. Saucier.  Mini-markers: a brief version of Goldberg's unipolar big-five markers. , 1994, Journal of personality assessment.

[14]  Gerald V. Barrett,et al.  Validity of Personnel Decisions: A Conceptual Analysis of the Inferential and Evidential Bases , 1989 .

[15]  F. Drasgow,et al.  Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. , 1993 .

[16]  P. A. Gade,et al.  The Measurement and Consequences of Military Organizational Commitment in Soldiers and Spouses , 2003 .

[17]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[18]  A. Panter,et al.  Two Approaches for Identifying Shared Personality Dimensions Across Methods , 1999 .

[19]  R. Landis,et al.  Equivalence of constructs measured by job-specific and commercially available aptitude tests , 1992 .

[20]  W Arthur,et al.  Relationships among personnel tests, age, and job performance. , 1990, Experimental aging research.

[21]  K. L. Evans.  Estimating Personality Constructs from Archival Data , 1997 .

[22]  S. West,et al.  The investigation of personality structure: Statistical models , 1997 .

[23]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[24]  L. Porter,et al.  The Measurement of Organizational Commitment. , 1979 .

[25]  R. MacCallum,et al.  Commentary on Quantitative Methods in I/O Research , 1998 .

[26]  Mark R. Leary,et al.  Introduction to behavioral research methods, 4th ed. , 2004 .

[27]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[28]  J. P. Wanous,et al.  Overall job satisfaction: how good are single-item measures? , 1997, The Journal of applied psychology.

[29]  Timothy R. Hinkin,et al.  A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires , 1998 .

[30]  B. Bracken,et al.  State of the Art Procedures for Translating, Validating and Using Psychoeducational Tests in Cross-Cultural Assessment , 1991 .

[31]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[32]  John P. Meyer,et al.  Testing the "Side-Bet Theory" of Organizational Commitment: Some Methodological Considerations , 1984 .

[33]  M. Browne.  Factor analysis of multiple batteries by maximum likelihood , 1980 .

[34]  Ian R. Gellatly,et al.  Affective and continuance commitment to the organization: Evaluation of measures and analysis of concurrent and time-lagged relations. , 1990 .

[35]  Stephanie C Payne,et al.  Opening Organizational Archives to Research: Analog Measures of Organizational Commitment , 2003 .

[36]  H. G. Osburn,et al.  Construct equivalence as an approach to replacing validated cognitive ability selection tests. , 1989 .

[37]  Alexander W. Austin.  The American Freshman: National Norms for Fall 1974. , 1981 .

[38]  M. Hagen,et al.  Ethical principles of psychologists and code of conduct. , 2002, The American psychologist.

[39]  Michael J. Zickar,et al.  THE PEN‐BASED COMPUTER AS AN ALTERNATIVE PLATFORM FOR TEST ADMINISTRATION , 1996 .

[40]  John P. Meyer,et al.  AFFECTIVE, CONTINUANCE, AND NORMATIVE COMMITMENT TO THE ORGANIZATION: A META-ANALYSIS OF ANTECEDENTS, CORRELATES, AND CONSEQUENCES , 2002 .

[41]  S. Messick.  Test validity and the ethics of assessment. , 1980 .

[42]  J. R. Larson,et al.  Research strategies and tactics in industrial and organizational psychology. , 1990 .

[43]  D. A. Kenny,et al.  Analysis of the multitrait^multimethod matrix by confirmatory factor analysis , 1992 .

[44]  M. Browne.  The maximum‐likelihood solution in inter‐battery factor analysis , 1979 .

[45]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[46]  J. Greenhaus,et al.  NEGATIVE AFFECTIVITY, ROLE STRESS, AND WORK-FAMILY CONFLICT , 2002 .

[47]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[48]  J. S. Long,et al.  Common Problems/Proper Solutions: Avoiding Error in Quantitative Research , 1988 .

[49]  Timothy R. Hinkin,et al.  An Analysis of Variance Approach to Content Validation , 1999 .

[50]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[51]  R. Cudeck.  Methods For Estimating Between-Battery Factors , 1982 .