Don't Let the Truth Get in the Way of a Good Story: An Illustration of Citation Bias in Epidemiologic Research

The production of scientific knowledge is susceptible to bias at every stage of the process, from what questions are asked by the investigator, to which method is chosen to gather data, to which analyses are conducted (e.g., “P-hacking,” wherein the method of statistical analysis and the degrees of freedom are manipulated until they yield statistically significant results) (1). Even after completion of a study, authors sometimes choose not to submit their work for publication because they are not satisfied with the results (i.e., the “file drawer” problem) (1), or they encounter difficulties with getting results published because of reviewer or editorial bias (“publication bias”) (2–4). Although prepublication biases have been well described in epidemiologic textbooks, postpublication biases, such as selective citation, have been less well documented. “Citation bias” occurs when scientists selectively cite papers based upon risk estimates that conform to their preconceived notions (5). When researchers have a bias in favor of “X causing Y,” they are more likely to cite papers that found evidence to support their view. Conversely, when researchers harbor a bias against a hypothesized association, they may selectively cite papers that report null findings. Here, we use research on job strain and the risk of coronary heart disease to examine factors that influence citations in peer-reviewed literature. In addition to the risk estimate for job strain relative to no job strain in each study, we take into account the impact factor of the publishing journal, which is an indicator of its prestige.