Selecting the best system: a decision-theoretic approach

The problem of selecting the best system from a nite set of alternatives is considered from a Bayesian decision-theoretic perspective. The framework presented is quite general, and permits selection from two or more systems, with replications that use either independent or common random numbers, with unknown mean and covariance for the output, and permits Gaussian or non-Gaussian simulation output. For the case of unknown means and variance with common random numbers, the framework provides a probability of correct selection that does not suer from problems associated with the Bonferroni inequality. We indicate some criteria for which the Bayesian approach and other approaches are in general agreement, or disagreement. The probability of correct selection can be calculated either by quadrature or by Monte Carlo simulation from the posterior distribution of the parameters of the statistical distribution of the simulation output. We also comment on expected-value decision-making versus optimization criteria based on other functionals of the distribution of the output.