The Usefulness of Unit Weights in Creating Composite Scores

Combining/weighting subscores into an aggregate score involves issues that apply to many fields in the organizational sciences (e.g., weighting predictors in selection, weighting multiple performance appraisal indicators, overall evaluation of organizations). The weights that are used in practice can be different (differential weights) or equal (unit weights). Relevant literature across multiple disciplines and multiple decades is reviewed. The literature indicates that unit weights have substantial predictive validity when compared with regression weights, but there is a lack of data on how other differential weighting strategies (e.g., weights generated by subject matter experts) compare to unit weights. In response, a primary and a meta-analytic study are provided here. The recent literature also contains some potential criticisms of unit weights in regard to personnel selection and content validation—and those statements are evaluated. The data and findings indicate that unit weights can be a highly appropriate approach for weighting under many circumstances.

[1]  Robert M. Guion,et al.  Personnel assessment, selection, and placement. , 1991 .

[2]  Dan A. Biddle Adverse Impact and Test Validation: A Practitioner's Guide to Valid and Defensible Employment Testing , 2005 .

[3]  Freda Kemp Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences , 2003 .

[4]  Philip L. Roth,et al.  Correcting the Effect Size of d for Range Restriction and Unreliability , 2001 .

[5]  Thomas R. Carretta,et al.  In Top-Down Decisions, Weighting Variables does Not Matter: A Consequence of Wilks' Theorem , 1998 .

[6]  R. Dawes Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[7]  Nambury S. Raju,et al.  A Comparison of Five Methods for Combining Multiple Criteria into a Single Composite , 1982 .

[8]  Richard A. Reid,et al.  Analytical MS/OR Tools Applied to a Plant Closure , 1994 .

[9]  J. F. Byrnes,et al.  Costing Human Resources: The Financial Impact of Behavior in Organizations , 1984 .

[10]  Nambury S. Raju,et al.  Methodology Review: Estimation of Population Validity and Cross-Validity, and the Use of Equal Weights in Prediction , 1997 .

[11]  Donald P. Schwab,et al.  A cross-validated comparison of five models used to predict graduate admissions committee decisions. , 1976 .

[12]  C. H. Lawshe STATISTICAL THEORY AND PRACTICE IN APPLIED PSYCHOLOGY , 1969 .

[13]  Lewis R. Goldberg,et al.  Methodological critiques. Seer over sign: The first "good" example? , 1968 .

[14]  N C Boreham,et al.  Can past academic criteria predict students at risk of future failure? , 1988, Medical education.

[15]  Kenneth P. Yusko,et al.  THE ROLE OF COGNITIVE ABILITY IN THE SUBGROUP DIFFERENCES AND INCREMENTAL VALIDITY OF ASSESSMENT CENTER EXERCISES , 1998 .

[16]  Michael G. Aamodt,et al.  Comparison of Four Methods for Weighting Multiple Predictors , 1985 .

[17]  Nambury S. Raju,et al.  Accuracy of Population Validity and Cross-Validity Estimation: An Empirical Comparison of Formula-Based, Traditional Empirical, and Equal Weights Procedures , 1999 .

[18]  Fred S. Switzer,et al.  Outliers and Influential Cases: Handling those Discordant Contaminated Maverick Rogues , 2008 .

[19]  Julian C. Stanley,et al.  Differential Weighting: A Review of Methods and Empirical Studies1 , 1970 .

[20]  John P. Hausknecht,et al.  Applicant Reactions to Selection Procedures: An Updated Model and Meta-Analysis , 2004 .

[21]  Frank L. Schmidt,et al.  The Relative Efficiency of Regression and Simple Unit Predictor Weights in Applied Differential Psychology , 1971 .

[22]  Philip Bobko,et al.  Note on standardized regression estimators. , 1980 .

[23]  Robert L. Linn,et al.  Path analysis: Psychological examples. , 1970 .

[24]  F. Drasgow,et al.  Alternative weighting schemes for linear prediction , 1978 .

[25]  Jermoe E. Doppelt Watch Your Weights. , 1957 .

[26]  Marvin H. Trattner COMPARISON OF THREE METHODS FOR ASSEMBLING APTITUDE TEST BATTERIES , 1963 .

[27]  Ward Edwards,et al.  Evaluating credit applications: A validation of multiattribute utility weight elicitation techniques , 1983 .

[28]  Jacob Cohen,et al.  THINGS I HAVE LEARNED (SO FAR) , 1990 .

[29]  John E. Hunter,et al.  Statistical power in criterion-related validation studies. , 1976 .

[30]  Howard Wainer,et al.  Estimating Coefficients in Linear Models: It Don't Make No Nevermind , 1976 .

[31]  Robert D. Gatewood,et al.  Human Resource Selection , 1997 .

[32]  R. Hogarth,et al.  Unit weighting schemes for decision making , 1975 .

[33]  P. Bobko Correlation and Regression: Applications for Industrial Organizational Psychology and Management , 2001 .

[34]  John G. Claudy A Comparison of Five Variable Weighting Procedures , 1972 .

[35]  Douglas W. Bray,et al.  Validation of employment tests for telephone company installation and repair occupations. , 1970 .

[36]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[37]  Dennis Doverspike,et al.  DEVELOPMENT OF A JOB ANALYSIS-BASED PROCEDURE FOR WEIGHTING AND COMBINING CONTENT RELATED TESTS INTO A SINGLE TEST BATTERY SCORE , 1996 .

[38]  S. S. Wilks Weighting systems for linear functions of correlated variables when there is no dependent variable , 1938 .

[39]  Dennis Doverspike,et al.  Simulations as a Method of Illustrating the Impact of Differential Weights on Personnel Selection Outcomes , 1996 .

[40]  James E. Laughlin Comment on "Estimating coefficients in linear models: It don't make no nevermind." , 1978 .