The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed.
BACKGROUND AND OBJECTIVE Publication bias and other sample size effects are issues for meta-analyses of test accuracy, as for randomized trials. We investigate limitations of standard funnel plots and tests when applied to meta-analyses of test accuracy and look for improved methods. METHODS Type I and type II error rates for existing and alternative tests of sample size effects were estimated and compared in simulated meta-analyses of test accuracy. RESULTS Type I error rates for the Begg, Egger, and Macaskill tests are inflated for typical diagnostic odds ratios (DOR), when disease prevalence differs from 50% and when thresholds favor sensitivity over specificity or vice versa. Regression and correlation tests based on functions of effective sample size are valid, if occasionally conservative, tests for sample size effects. Empirical evidence suggests that they have adequate power to be useful tests. When DORs are heterogeneous, however, all tests of funnel plot asymmetry have low power. CONCLUSION Existing tests that use standard errors of odds ratios are likely to be seriously misleading if applied to meta-analyses of test accuracy. The effective sample size funnel plot and associated regression test of asymmetry should be used to detect publication bias and other sample size related effects.
In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias.
BACKGROUND AND OBJECTIVE Publication bias and related biases can lead to overly optimistic conclusions in systematic reviews. The funnel plot, which is frequently used to detect such biases, has not yet been subjected to empirical evaluation as a visual tool. We sought to determine whether researchers can correctly identify publication bias from visual inspection of funnel plots in typical-size systematic reviews. METHODS A questionnaire with funnel plots containing 10 studies each (the median number in medical meta-analyses) was completed by 41 medical researchers, including clinical research fellows in a meta-analysis class, faculty in clinical care research, and experienced systematic reviewers. RESULTS On average, participants correctly identified 52.5% (95% CI 50.6-54.4%) of the plots as being affected or unaffected by publication bias. The weighted mean percent correct, which adjusted for the fact that asymmetric plots are more likely to occur in the presence of publication bias, was also low (48.3 to 62.8%, depending on the presence or absence of publication bias and heterogeneous study effects). CONCLUSION Researchers who assess for publication bias using the funnel plot may be misled by its shape. Authors and readers of systematic reviews need to be aware of the limitations of the funnel plot.
time series recurrent neural network metric space health care discrete wavelet transform sample size confidence interval discrete fourier transform systematic review dimensionality reduction internet service euclidean distance traffic engineering internet service provider web search engine amino acid internet traffic intensive care unit time warping similarity search background and objective x-ray computed tomography heart failure traffic classification large time body mass index early diagnosi evaluation procedure dimensionality reduction technique growth factor internet routing kidney disease signal transduction symmetric encryption chronic kidney disease sequence database chronic kidney time series database today internet scaling behavior internet backbone searchable symmetric encryption cardiac surgery series database internet traffic classification searchable symmetric oxidative stres publication bia cell surface efficient similarity external validation large time series efficient similarity search time warping distance glomerular filtration rate effective sample size hospital admission fast similarity fast similarity search plasma membrane acute kidney injury acute kidney kidney injury internet traffic engineering approximate similarity search search in large dynamic searchable area under curve kidney transplantation today internet traffic sse scheme dynamic searchable symmetric radical polymerization abbott laboratory traffic classification technique renal replacement therapy cell physiology ckd patient wide-area internet internet traffic measurement improved definition fibroblast growth factor chain transfer biological marker fibroblast growth genetic heterogeneity lipid raft excretory function cns disorder entity name part qualifier - adopted cessation of life standards characteristic complement system protein one thousand hypertensive disease limited stage (cancer stage) tissue membrane glutathione s-transferase adverse reaction to drug diameter (qualifier value) congenital abnormality kidney failure, chronic renal insufficiency creatinine measurement, serum (procedure) forecast of outcome stage level 1 microgram per liter milliliter per minute diagnosis, clinical vesicle (morphologic abnormality) lipid metabolism disorder transplanted tissue membrane protein traffic stage level 3 cfh gene hemolytic-uremic syndrome kidney failure, acute blighia sapida creatinine clearance measurement cystatin c (substance) stage level 5