Number of Replications Required in Monte Carlo Simulation Studies: A Synthesis of Four Studies

Monte Carlo simulations are used extensively to study the performance of statistical tests and control charts. Researchers have used various numbers of replications, but rarely provide justification for their choice. Currently, no empirically-based recommendations regarding the required number of replications exist. Twenty-two studies were re-analyzed to determine empirically-based recommendations.

[1]  Michael B. C. Khoo,et al.  Incorporating Runs Rules into Hotelling's χ2 Control Charts , 2003 .

[2]  Michael B. C. Khoo,et al.  Computing the Percentage Points of the Run-Length Distributions of Multivariate CUSUM Control Charts , 2002 .

[3]  Joel R. Levin,et al.  New developments in pairwise multiple comparisons : some powerful and practicable procedures , 1991 .

[4]  Clifford M. Hurvich,et al.  The impact of model selection on inference in linear regression , 1990 .

[5]  A S Neubauer,et al.  The EWMA control chart: properties and comparison with other quality-control procedures by computer simulation. , 1997, Clinical chemistry.

[6]  Morton B. Brown,et al.  The Small Sample Behavior of Some Statistics Which Test the Equality of Several Means , 1974 .

[7]  Sam Weerahandi,et al.  Size performance of some tests in one-way anova , 1998 .

[8]  Fah Fatt Gan,et al.  Shewhart Charts for Monitoring the Variance Components , 2004 .

[9]  B. L. Welch.  ON THE COMPARISON OF SEVERAL MEAN VALUES: AN ALTERNATIVE APPROACH , 1951 .

[10]  Michael B. C. Khoo,et al.  Performance Measures for the Shewhart Control Chart , 2004 .

[11]  Joe Saur.  Review of "SASÒ for monte carlo studies: a guide for quantitative researchers by Xitao Fan, Ákos Felsvlyi, Stephen A. Sivo, and Sean C. Keenan." SAS Institute, Inc. 2002 , 2004, SOEN.

[12]  Subhabrata Chakraborti,et al.  A nonparametric control chart based on the Mann-Whitney statistic , 2003 .

[13]  Welch Bl.  THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[14]  William H. Fellner.  Algorithm AS 258: Average Run Lengths for Cumulative Sum Schemes , 1990 .

[15]  Xitao Fan,et al.  SAS for Monte Carlo studies : a guide for quantitative researchers , 2002 .

[16]  Philip H. Ramsey.  Comparison of closed testing procedures for pairwise testing of means. , 2002, Psychological methods.

[17]  Michael B. C. Khoo.  Increasing the Sensitivity of Control Chart for Fraction Nonconforming , 2003 .

[18]  A.B. Dumont,et al.  Quality in Engineering , 1950, Proceedings of the IRE.

[19]  B. L. Welch.  The generalisation of student's problems when several different population variances are involved. , 1947, Biometrika.

[20]  T. E. Dielman,et al.  Testing normality of regression disturbances: a Monte Carlo study of the Filliben test , 1991 .

[21]  V. Flack,et al.  Frequency of Selecting Noise Variables in Subset Regression Analysis: A Simulation Study , 1987 .

[22]  T. Morikawa,et al.  Power evaluation of various modified Bonferroni procedures by a Monte Carlo study. , 1996, Journal of biopharmaceutical statistics.

[23]  Leslie Godfrey,et al.  Testing for multiplicative heteroskedasticity , 1978 .

[24]  S. Olejnik,et al.  Power of Pairwise Multiple Comparisons in the Unequal Variance Case. , 1994 .

[25]  Myoung-Jin Kim,et al.  Number of Replications Required in Control Chart Monte Carlo Simulation Studies , 2007, Commun. Stat. Simul. Comput..

[26]  W. Griffiths,et al.  A MONTE CARLO EVALUATION OF THE POWER OF SOME TESTS FOR HETEROSCEDASTICITY , 1986 .

[27]  Jamie D. Mills,et al.  Using Wherry's Adjusted R 2 and Mallow's Cp for Model Selection From All Possible Regressions , 2000 .