A Class of Optimum Improtance Sampling Strategies

Abstract The problem of estimating expectations of functionals of random vectors via simulation is investigated. As an alternative to simple averaging, an efficient form of Monte Carlo simulations known as Importance Sampling can be employed to produce arbitrarily accurate estimates of the expectation of interest. The fundamental problem in Importance Sampling is to determine the appropriate statistics under which to conduct the simulation. These statistics are typically determined from a constraint class by minimizing the variance of the Importance Sampling estimator over the class of admissible statistics. In previous work, the authors showed that minimizing the Importance Sampling variance is equivalent to finding the distribution which is closest to the unconstrained optimal distribution with respect to a specific Information-Theoretic f -divergence or Ali-Silvey distance. In this work, the authors determine the optimal biasing density from a constraint class whose controlling parameter is fundamental in the performance analysis of Importance Sampling. In addition, it is shown that the constrained optimal distribution from this class minimizes every statistical distance measure to the global optimal distribution and as a consequence is optimal with respect to a large family of nonlinear cost functions. Salient features of this distribution are: (1) unlike the unconstrained optimal solution, this distribution can be made independent of the parameter to be estimated and thus can admit to implementation, and (2) this distribution renders performance gains which can be made arbitrarily close to the optimal gains. Applications to estimating probabilities of rare events (e.g., error rates in communication systems) will be presented. Further analysis will show that in this case, the savings over Monte Carlo simulations become unbounded as the probability of the rare event diminishes.