A Test of Two Refinements in Procedures for Meta-Analysis

This study used Monte Carlo simulation to examine the increase in accuracy resulting from 2 statistical refinements of the interactive Schmidt-Hunter procedures for meta-analysis: the use of the mean correlation instead of individual correlations in the estimation of sampling error variance, and a procedure that takes into account the nonlinear nature of the range-restriction correction. In all of the cases examined, these refinements increased the accuracy of the interactive procedure in estimating the variance of population correlations and resulted in more accuracy than other procedures examined. The use of the mean correlation in the sampling error variance formula also increased the accuracy of variance estimates for the multiplicative and Taylor Series procedures. Meta-analysis has become a widely used research technique in behavioral science. Among the different available meta-analytic approaches, the artifact-distribution-based meta-analytic methods for correlation coefficients are used most often, with numerous applications of this technique in different areas of behavioral science (Hunter & Schmidt, 1990; Schmidt, 1992). Computer simulation studies have examined the accuracy of procedures of this type (Callender & Osburn, 1980; Kemery, Mossholder, & Roth, 1987; Mendosa & Reinhardt, 1991; Raju & Burke, 1983). Most applications of meta-analysis to large real-world databases have yielded positive nonzero estimates of the variance of population correlations (e.g., Pearlman, Schmidt, & Hunter, 1980), although the variance estimates have generally been very small. Simulations (e.g., see Raju & Burke, 1983) have shown that although all of the mean population correlation estimates were very accurate, all of the procedures except the Schmidt-Hunter noninteractive procedure overestimated the true variance of population correlations. As a result, there have been continuous efforts to improve the accuracy of the existing meta-analytic procedures and to explain why they overestimate the true variance of population correlations. This study focused on two major modifications of the Schmidt-Hunter noninteractive (Pearlman et al., 1980; Schmidt, Hunter, Pearlman, & Shane, 1979) and interactive (Schmidt, Gast-Rosenberg, & Hunter, 1980) procedures for meta-analysis and used computer simulations to test the estimates produced by these improved procedures. We hypothesized that with these improvements overestimation of true variance of population correlations across studies would be reduced and that more accurate estimates of the population parameters