Meta-analyses of hydroxyethyl starch for volume resuscitation.

To the Editor: Most textbooks and guidance documents on systematic reviews discuss the caveats and potential risks of heterogeneity in meta-analysis. It is not appreciated that extreme homogeneity in a meta-analysis can be a far stronger signal of major problems. The careful meta-analysis of hydroxyethyl starch for fluid resuscitation by Dr Zarychanski and colleagues exemplifies this issue. The Cochrane review on this topic showed extreme between-study homogeneity (left-sided P=.001). When my team screened the entire Cochrane database to identify metaanalyses in which the results across the included studies were too homogeneous, that meta-analysis stood out and we communicated with the principal investigator (Dr Boldt) to find out how he had found such identical results across 5 seemingly different randomized trials. The author indicated that these were not overlapping data sets but that they did represent independent trials performed at different periods at the same institution. In an article about extreme homogeneity, we discussed the homogeneity in that meta-analysis as potentially reflecting the performance of all trials at the same institution. In the discussion, we cautiously listed fraud as one potential reason for extremely homogeneous results. Now, many years later, it is widely recognized that this was one of the most prolific cases of fraud. The take-home message is that meta-analyses that include multiple studies with practically identical results should be viewed with great caution. Fabrication, double publication, plagiarism, variants of salami publication, allegiance or confirmation bias, or other questionable practices may sometimes be involved. In particular, when multiple studies in a meta-analysis come from the same team of investigators (or their affiliates) and all of them find exactly the same conclusion, it is prudent to question and try to understand why so much evidence was generated by a single team. Science is an open global enterprise, and intense inbreeding may signal scientific fields in which one or a few stakeholders simply manipulate the published evidence according to their wishes and beliefs. In these cases, independence of the studies cannot be taken for granted and homogeneity, especially extreme homogeneity, may be a sign of major trouble for a meta-analysis rather than a reassurance of consistency.