IMP: A decision aid for multiattribute evaluation using imprecise weight estimates

In multicriteria analysis, weights are inferred from judgements. Judgements are subject to uncertainty and the weights found depend on the method used so that weights must necessarily be imprecise. Describing weights by a suitable probability distribution captures this imprecision and leads to correspondingly imprecise estimates of the multiattribute scores given to alternatives. A method is described which enables the parameters of a Dirichlet distribution to be found from three point weight estimates and for this to be used to determine whether or not alternatives may justifiably be differentiated by their scores. A simple chart shows this information and permits easy interaction. An illustration shows the effects of different sources of imprecision and of aggregating individual assessments in a group.

[1]  Robin M. Hogarth,et al.  Cognitive Processes and the Assessment of Subjective Probability Distributions , 1975 .

[2]  Ian N. Durbach,et al.  An experimental study of the effect of uncertainty representation on decision making , 2011, Eur. J. Oper. Res..

[3]  Francesco Orilia,et al.  Semantics and Cognition , 1991 .

[4]  José María Moreno-Jiménez,et al.  Reciprocal distributions in the analytic hierarchy process , 2000, Eur. J. Oper. Res..

[5]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[6]  David B. Paradice,et al.  The Effects of Integrating Cognitive Feedback and Multi-attribute Utility-Based Multicriteria Decision-Making Methods in GDSS , 1999 .

[7]  Indrani Basak,et al.  Probabilistic judgments specified partially in the analytic hierarchy process , 1998, Eur. J. Oper. Res..

[8]  M. Cowles,et al.  On the Origins of the .05 Level of Statistical Significance , 1982 .

[9]  A. Diederich,et al.  Evaluating and Combining Subjective Probability Estimates , 1997 .

[10]  Risto Lahdelma,et al.  SMAA-2: Stochastic Multicriteria Acceptability Analysis for Group Decision Making , 2001, Oper. Res..

[11]  R. Hämäläinen,et al.  Preference programming through approximate ratio comparisons , 1995 .

[12]  J. Barnes.  Science and Speculation Studies in Hellenistic Theory and Practice /Edited by Jonathan Barnes ... [Et Al.]. --. -- , 1982 .

[13]  Eduardo Fernández,et al.  Handling multicriteria preferences in cluster analysis , 2010, Eur. J. Oper. Res..

[14]  Alfonso Mateos,et al.  Monte Carlo Simulation Techniques in a Decision Support System for Group Decision Making , 2005 .

[15]  Gregg C. Oden,et al.  Integration of fuzzy logical information. , 1977 .

[16]  J. Figueira,et al.  A survey on stochastic multicriteria acceptability analysis methods , 2008 .

[17]  Kaisa Miettinen,et al.  Ordinal criteria in stochastic multicriteria acceptability analysis (SMAA) , 2003, Eur. J. Oper. Res..

[18]  João Paulo Costa,et al.  Visual and Interactive Comparative Analysis of Individual Opinions: A Group Decision Support Tool , 2014 .

[19]  Eugene D. Hahn.  Decision Making with Uncertain Judgments: A Stochastic Formulation of the Analytic Hierarchy Process , 2003, Decis. Sci..

[20]  Luis V. García,et al.  Escaping the Bonferroni iron claw in ecological studies , 2004 .

[21]  K. Hipel,et al.  A case-based distance method for screening in multiple-criteria decision aid ☆ , 2008 .

[22]  Alyson G. Wilson,et al.  Cognitive Factors Affecting Subjective Probability Assessment , 1994 .

[23]  I. Erev,et al.  Verbal versus numerical probabilities: Efficiency, biases, and the preference paradox☆ , 1990 .

[24]  D. Kahneman,et al.  Heuristics and Biases: The Psychology of Intuitive Judgment , 2002 .

[25]  Peter Naudé,et al.  The Stability of Judgmental Modelling: an Application in the Social Services , 1998 .

[26]  P. Curșeu,et al.  The Effects of Framing on Inter-group Negotiation , 2008 .

[27]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[28]  W. Bottom.  Heuristics and Biases: The Psychology of Intuitive JudgmentHeuristics and Biases: The Psychology of Intuitive Judgment, edited by Gilovich Thomas Griffin Dale Kahneman Daniel. Cambridge: Cambridge University Press, 2002. , 2004 .

[29]  M. Moran.  Arguments for rejecting the sequential Bonferroni in ecological studies , 2003 .

[30]  Craig R. Fox,et al.  The Role of Context in the Communication of Uncertain Beliefs , 1998 .

[31]  B. Fischhoff,et al.  Knowing with Certainty: The Appropriateness of Extreme Confidence. , 1977 .

[32]  José María Moreno-Jiménez,et al.  AHP-Group Decision Making: A Bayesian Approach Based on Mixtures for Group Pattern Identification , 2007 .

[33]  Robert L. Winkler,et al.  Uncertainty in probabilistic risk assessment , 1996 .

[34]  T. Stewart,et al.  A comparison of simplified value function approaches for treating uncertainty in multi-criteria decision analysis , 2012 .

[35]  Joseph J. Moder,et al.  Judgment Estimates of the Moments of Pert Type Distributions , 1968 .

[36]  D. Budescu,et al.  A Psychometric Analysis of the "Divide and Conquer" Principle in Multicriteria Decision Making. , 1998, Organizational behavior and human decision processes.

[37]  Edgar M. Johnson.  Numerical Encoding of Qualitative Expressions of Uncertainty , 1973 .

[38]  Andrew P. Sage,et al.  ARIADNE: A knowledge-based interactive system for planning and decision support , 1984, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  Raimo P. Hämäläinen,et al.  Decision Support by Interval SMART/SWING - Incorporating Imprecision in the SMART and SWING Methods , 2005, Decis. Sci..

[40]  D. Mccloskey,et al.  The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives , 2008 .

[41]  Gideon Keren,et al.  On The Calibration of Probability Judgments: Some Critical Comments and Alternative Perspectives , 1997 .

[42]  Reuven R. Levary,et al.  A simulation approach for handling uncertainty in the analytic hierarchy process , 1998, Eur. J. Oper. Res..

[43]  Amnon Rapoport,et al.  Measuring the Vague Meanings of Probability Terms , 1986 .

[44]  Elizabeth C. Hirschman,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[45]  John C. Butler,et al.  Simulation techniques for the sensitivity analysis of multi-criteria decision models , 1997 .

[46]  Alfonso Mateos,et al.  Missing consequences in multiattribute utility theory , 2009 .

[47]  I. Blanchette,et al.  The influence of affect on higher level cognition: A review of research on interpretation, judgement, decision making and reasoning , 2010 .

[48]  Risto Lahdelma,et al.  SMAA - Stochastic multiobjective acceptability analysis , 1998, Eur. J. Oper. Res..

[49]  Raimo P. Hämäläinen,et al.  Preference Assessment by Imprecise Ratio Statements , 1992, Oper. Res..

[50]  Jason R. W. Merrick.  Getting the Right Mix of Experts , 2008, Decis. Anal..

[51]  R. Ranyard,et al.  Chinese and English Speakers’ Linguistic Expression of Probability and Probabilistic Thinking , 1999 .

[52]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[53]  R. L. Winkler,et al.  Advances in Decision Analysis: Aggregating Probability Distributions , 2007 .

[54]  Alan Jessop,et al.  Using imprecise estimates for weights , 2011, J. Oper. Res. Soc..

[55]  Simon French,et al.  e-Participation and Decision Analysis , 2007, Decis. Anal..

[56]  John R. Doyle,et al.  A comparison of three weight elicitation methods: good, better, and best , 2001 .

[57]  H. Arkes,et al.  Assessing the Merits and Faults of Holistic and Disaggregated Judgments , 2009 .

[58]  Jiye Liang,et al.  Evaluation of the results of multi-attribute group decision-making with linguistic information , 2012 .

[59]  Dennis T. Kennedy,et al.  An Applied Study Using the Analytic Hierarchy Process to Translate Common Verbal Phrases to Numerical Probabilities , 1997 .

[60]  John G. Gray,et al.  Medicine , 1902, Glasgow Medical Journal.

[61]  D. Altman,et al.  Multiple significance tests: the Bonferroni method , 1995, BMJ.

[62]  Bruce E. Barrett,et al.  Decision quality using ranked attribute weights , 1996 .

[63]  D. O. Cooper,et al.  A Simple Way of Developing a Probability Distribution of Present Value , 1976 .

[64]  Jean-Marc Bernard,et al.  An introduction to the imprecise Dirichlet model for multinomial data , 2005, Int. J. Approx. Reason..

[65]  Jiju Antony,et al.  Application of stochastic analytic hierarchy process within a domestic appliance manufacturer , 2007, J. Oper. Res. Soc..

[66]  F. H. Barron,et al.  Selecting a best multiattribute alternative with partial information about attribute weights , 1992 .

[67]  Achille C. Varzi,et al.  Deviant Logic, Fuzzy Logic: Beyond the Formalism , 1996 .

[68]  T. Chavez,et al.  Modeling and measuring the effects of vagueness in decision models , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[69]  H. A. Donegan,et al.  A New Approach to Ahp Decision‐Making , 1992 .

[70]  Luis C. Dias,et al.  Dealing with imprecise information in group multicriteria decisions: a methodology and a GDSS architecture , 2005, Eur. J. Oper. Res..

[71]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[72]  Simon French,et al.  Interactive Multi-Objective Programming: Its Aims, Applications and Demands , 1984 .

[73]  E. S. Pearson,et al.  Approximate means and standard deviations based on distances between percentage points of frequency curves , 1965 .

[74]  Kwang-Jae Kim,et al.  Compromising prioritization from pairwise comparisons considering type I and II errors , 2010, Eur. J. Oper. Res..

[75]  Danielle R.M. Timmermans,et al.  The impact of task complexity on information use in multi‐attribute decision making , 1993 .

[76]  E. Rosenbloom.  A probabilistic interpretation of the final rankings in AHP , 1997 .

[77]  João Paulo Costa,et al.  The AGAP system: A GDSS for project analysis and evaluation , 2003, Eur. J. Oper. Res..

[78]  R. D. Holder,et al.  Some Comments on the Analytic Hierarchy Process , 1990 .

[79]  John R. Doyle,et al.  Comparing the validity of numerical judgements elicited by direct rating and point allocation: Insights from objectively verifiable perceptual tasks , 2013, Eur. J. Oper. Res..

[80]  Maya Bar-Hillel,et al.  On the subjective probability of compound events , 1973 .

[81]  Konstantinos Kirytopoulos,et al.  A Calibrated Group Decision Process , 2014 .

[82]  Charalambos L. Iacovou,et al.  Determining attribute weights using mathematical programming , 2003 .

[83]  Silja Renooij,et al.  Talking probabilities: communicating probabilistic information with words and numbers , 1999, Int. J. Approx. Reason..

[84]  Ramakrishnan Ramanathan,et al.  A qualitative perspective to deriving weights from pairwise comparison matrices , 2010 .

[85]  Yee Mey Goh,et al.  Approaches to displaying information to assist decisions under uncertainty , 2012 .

[86]  Risto Lahdelma,et al.  Two ways to handle dependent uncertainties in multi-criteria decision problems , 2009 .

[87]  Robert L. Winkler,et al.  Multiple Experts vs. Multiple Methods: Combining Correlation Assessments , 2004, Decis. Anal..

[88]  R. L. Winkler.  The Assessment of Prior Distributions in Bayesian Analysis , 1967 .

[89]  P. Tadikamalla,et al.  The Analytic Hierarchy Process in an uncertain environment: A simulation approach , 1996 .

[90]  Thomas S. Wallsten,et al.  Some Effects of Vocabulary and Communication Task on the Understanding and Use of Vague Probability Expressions , 1991 .

[91]  Max Henrion,et al.  Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis , 1990 .

[92]  P. Walley.  Inferences from Multinomial Data: Learning About a Bag of Marbles , 1996 .

[93]  Jian Ma,et al.  A vague set based decision support approach for evaluating research funding programs , 2013, Eur. J. Oper. Res..

[94]  Jyrki Kangas,et al.  Uncertainty in Expert Predictions of the Ecological Consequences of Forest Plans , 1996 .

[95]  Gilberto Montibeller,et al.  Supporting the allocation of software development work in distributed teams with multi-criteria decision analysis , 2008 .

[96]  J. Kadane,et al.  Experiences in elicitation , 1998 .

[97]  T. Perneger.  What's wrong with Bonferroni adjustments , 1998, BMJ.

[98]  Martin Weber,et al.  Behavioral influences on weight judgments in multiattribute decision making , 1993 .

[99]  Quirico Semeraro,et al.  A new method to cope with decision makers' uncertainty in the equipment selection process , 2004 .

[100]  D. Keefer,et al.  Better estimation of PERT activity time parameters , 1993 .