An analysis of five simulation methods for determining the number of replications in a complex Monte Carlo study

In this article we summarize the results of a simulation study that compares and contrasts five different methods for selecting the number of replications necessary to accurately estimate a group of parameters utilizing the simple Monte Carlo method. The five methods are: an extension to several dimensions of Stein's two-stage procedure, an extension of Hall's three-stage procedure, a purely-sequential procedure that continuously monitors the simulation output, a continuously-monitoring procedure with a correction term, and a maximum eigenvalue procedure due to Srivastava. This study varies three design factors: the dimension of the simulated response (two or three), the number of replications necessary to be accurate (small, moderate and large sample sizes), and the kind of accuracy sets (rectangular or spherical). The simulation methods are compared on the basis of their probability of being accurate and on the basis of their efficiency, as measured by the distribution of the number of replications needed to be accurate. Typical distributional and density estimates are presented in both table and graph form.