Subjective Versus Statistical Importance Weights: A Criterion Validation.

Abstract : The present paper proposes a research paradigm for comparing weight estimates to empirically derived 'true' weights, thus obtaining a measure of the criterion validity of different weight estimation techniques. Subjects are first taught a multi-attribute utility (MAU) model via multiple-cue probability learning (MCPL) and outcome feedback. Then, various assessments of the importance weight parameters for the model attributes are obtained. Composites formed from these weights are subsequently compared to composites formed from optimal statistical weights derived from outcome feedback. Data are reported from 17 subjects who were taught one of three 'diamond worth' MAU models in 100 feedback trials. The models all involved four attributes (cut, color, clarity, and carat weight), and varied in the 'environmental correlations' among the dimensions (either (1) all uncorrelated, (2) one large positive correlation, or (3) two large negative correlations). The results of the present study are discussed from both an applied and theoretical perspective. To the decision analyst in the field, the present results give support to the belief that the parameter esimates obtained from clients define a 'true' normative preference function. Theoretically, the findings of this study are strong evidence that people are aware of their own cognitive processes.