Guessing preferences: A new approach to multi-attribute ranking and selection

We consider an analyst tasked with using simulation to help a decision-maker choose among several decision alternatives. Each alternative has several competing attributes, e.g., cost and quality, that are unknown but can be estimated through simulation. We model this problem in a Bayesian context, where the decision-maker's preferences are described by a utility function, but this utility function is unknown to the analyst. The analyst must choose how to allocate his simulation budget among the alternatives in the face of uncertainty about both the alternatives' attributes, and the decision-maker's preferences. Only after simulation is complete are the decision-maker's preferences revealed. In this context, we calculate the value of the information in simulation samples, and propose a new multi-attribute ranking and selection procedure based on this value. This procedure is able to incorporate prior information about the decision-maker's preferences to improve sampling efficiency.

[1]  Loo Hay Lee,et al.  Stochastic Simulation Optimization - An Optimal Computing Budget Allocation , 2010, System Engineering and Operations Research.

[2]  Peter I. Frazier,et al.  Knowledge-Gradient Methods for Statistical Learning , 2009 .

[3]  F. B. Vernadat,et al.  Decisions with Multiple Objectives: Preferences and Value Tradeoffs , 1994 .

[4]  Jürgen Branke,et al.  Sequential Sampling to Myopically Maximize the Expected Value of Information , 2010, INFORMS J. Comput..

[5]  Chun-Hung Chen,et al.  Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization , 2000, Discret. Event Dyn. Syst..

[6]  Ali E. Abbas,et al.  Entropy methods for adaptive utility elicitation , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[7]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[8]  Daphne Koller,et al.  Making Rational Decisions Using Adaptive Utility Elicitation , 2000, AAAI/IAAI.

[9]  M. Kuhl,et al.  APPLICATION OF MULTI-OBJECTIVE SIMULATION-OPTIMIZATION TECHNIQUES TO INVENTORY MANAGEMENT PROBLEMS , 2005 .

[10]  Jason R. W. Merrick.  Bayesian Simulation and Decision Analysis: An Expository Survey , 2009, Decis. Anal..

[11]  Louis Anthony Cox,et al.  Wiley encyclopedia of operations research and management science , 2011 .

[12]  R. Kane,et al.  Methodology for measuring health-state preferences--II: Scaling methods. , 1989, Journal of clinical epidemiology.

[13]  Warren B. Powell,et al.  The knowledge-gradient stopping rule for ranking and selection , 2008, 2008 Winter Simulation Conference.

[14]  Douglas J. Morrice,et al.  Ranking and Selection with Multiple "Targets" , 2006, Proceedings of the 2006 Winter Simulation Conference.

[15]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[16]  S. Andradóttir,et al.  Fully sequential procedures for comparing constrained systems via simulation , 2010 .

[17]  Sigrún Andradóttir,et al.  Finding the best in the presence of a stochastic constraint , 2005, Proceedings of the Winter Simulation Conference, 2005..

[18]  Stephen E. Chick,et al.  New Two-Stage and Sequential Procedures for Selecting the Best Simulated System , 2001, Oper. Res..

[19]  David Lindley,et al.  Optimal Statistical Decisions , 1971 .

[20]  Warren B. Powell,et al.  The Knowledge-Gradient Policy for Correlated Normal Beliefs , 2009, INFORMS J. Comput..

[21]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[22]  D. Winterfeldt,et al.  Comparison of weighting judgments in multiattribute utility measurement , 1991 .

[23]  Craig Boutilier,et al.  A POMDP formulation of preference elicitation problems , 2002, AAAI/IAAI.

[24]  L. Lee,et al.  Finding the non-dominated Pareto set for multi-objective simulation models , 2010 .

[25]  Loo Hay Lee,et al.  Finding the pareto set for multi-objective simulation models by minimization of expected opportunity cost , 2007, 2007 Winter Simulation Conference.

[26]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[27]  Loo Hay Lee,et al.  A multi-objective selection procedure of determining a Pareto set , 2009, Comput. Oper. Res..

[28]  Stephen E. Chick,et al.  Economic Analysis of Simulation Selection Problems , 2009, Manag. Sci..

[29]  S. Gupta,et al.  Bayesian look ahead one-stage sampling allocations for selection of the best population , 1996 .

[30]  Warren B. Powell,et al.  A Knowledge-Gradient Policy for Sequential Information Collection , 2008, SIAM J. Control. Optim..

[31]  Stephen E. Chick,et al.  New Procedures to Select the Best Simulated System Using Common Random Numbers , 2001, Manag. Sci..

[32]  Jürgen Branke,et al.  New greedy myopic and existing asymptotic sequential selection procedures: preliminary empirical results , 2007, 2007 Winter Simulation Conference.