On the convergence of multiattribute weighting methods

Abstract The convergent validity of five multiattribute weighting methods is studied in an Internet experiment. This is the first experiment where the subjects created the alternatives and attributes themselves. Each subject used five methods to assess attribute weights – one version of the analytic hierarchy process (AHP), direct point allocation, simple multiattribute rating technique (SMART), swing weighting, and tradeoff weighting. They can all be used following the principles of multiattribute value theory. Furthermore, SMART, swing, and AHP ask the decision makers to give directly the numerical estimates of weight ratios although the elicitation questions are different. In earlier studies these methods have yielded different weights. Our results suggest that the resulting weights are different because the methods explicitly or implicitly lead the decision makers to choose their responses from a limited set of numbers. The other consequences from this are that the spread of weights and the inconsistency between the preference statements depend on the number of attributes that a decision maker considers simultaneously.

[1]  Raimo P. Hämäläinen,et al.  Preference Assessment by Imprecise Ratio Statements , 1992, Oper. Res..

[2]  G. W. Fischer.  Range Sensitivity of Attribute Weights in Multiattribute Value Models , 1995 .

[3]  W. Edwards.  How to Use Multi-Attribute Utility Measurement for Social Decision Making , 1976 .

[4]  J. Dombi,et al.  A method for determining the weights of criteria: the centralized weights , 1986 .

[5]  L. Beach,et al.  Do ranks suffice? A comparison of alternative weighting approaches in value elicitation , 1995 .

[6]  R. Hämäläinen,et al.  An Experiment on the Numerical Modelling of Verbal Ratio Statements , 1997 .

[7]  H. Raiffa,et al.  Decisions with Multiple Objectives , 1993 .

[8]  T. Saaty.  Highlights and critical points in the theory and application of the Analytic Hierarchy Process , 1994 .

[9]  V. Belton.  A comparison of the analytic hierarchy process and a simple multi-attribute value function , 1986 .

[10]  David L. Olson,et al.  Consistency and Accuracy in Decision Aids: Experiments with Four Multiattribute Systems* , 1995 .

[11]  Ahti Salo.  Inconsistency analysis by approximately specified priorities , 1993 .

[12]  R. Hämäläinen,et al.  Notes on the weighting biases in value trees , 1998 .

[13]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[14]  Martin Weber,et al.  Behavioral influences on weight judgments in multiattribute decision making , 1993 .

[15]  Michael H. Birnbaum,et al.  Problems with So-Called “Direct” Scaling , 1982 .

[16]  D. Winterfeldt,et al.  Comparing Hierarchical and Nonhierarchical Weighting Methods for Eliciting Multiattribute Value Models , 1987 .

[17]  William C. Wedley,et al.  Alternative Scales in AHP , 1989 .

[18]  David L. Olson,et al.  Implementation of the centroid method of Solymosi and Dombi , 1992 .

[19]  Guk-Rwang Won.  American Society for Testing and Materials , 1987 .

[20]  Martin Weber,et al.  The Effect of Attribute Ranges on Weights in Multiattribute Utility Measurements , 1993 .

[21]  F. Lootsma.  SCALE SENSITIVITY IN THE MULTIPLICATIVE AHP AND SMART , 1993 .

[22]  Valerie Belton,et al.  On a short-coming of Saaty's method of analytic hierarchies , 1983 .

[23]  Ward Edwards,et al.  How to Use Multiattribute Utility Measurement for Social Decisionmaking , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[24]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .

[25]  D. Winterfeldt,et al.  Comparison of weighting judgments in multiattribute utility measurement , 1991 .

[26]  Jt Kuznicki,et al.  Selected Sensory Methods: Problems and Approaches to Measuring Hedonics , 1982 .

[27]  E. Choo,et al.  A UNIFIED APPROACH TO AHP WITH LINKING PINS , 1993 .

[28]  R. Hämäläinen,et al.  On the measurement of preferences in the analytic hierarchy process , 1997 .

[29]  T. Stewart.  A CRITICAL SURVEY ON THE STATUS OF MULTIPLE CRITERIA DECISION MAKING THEORY AND PRACTICE , 1992 .

[30]  C. C. Waid,et al.  An Experimental Comparison of Different Approaches to Determining Weights in Additive Utility Models , 1982 .