Translating Scientific Discovery Into Health Policy Impact: Innovative Bibliometrics Bridge Translational Research Publications to Policy Literature

To understand how translational science efforts lead to outcomes, it is common to examine publications as a key step in the translational process. The National Institutes of Health’s Clinical and Translational Science Awards (CTSA) program aims to accelerate that process by providing support to investigators. Although it is challenging to measure the impact of such support on translational outcomes, CTSA-supported research that arises in research publications can advance translation through use of these publications in public policy and guideline documents from government health agencies, intergovernmental organizations, and other outlets. Using cutting-edge bibliometric tools, the authors evaluated how CTSA-supported research has extended its impact beyond academic silos to influence public policy literature. The authors identified approximately 118,490 publications that acknowledged receiving support from a CTSA hub, from the inception of the program in 2006 through 2021. Articles were queried in the Overton policy database, which indexes references to publications in global policy literature. The search revealed 13% of CTSA-supported articles were referenced in policy documents, significantly more than the expected proportion (10%) calculated by Overton. References came from 576 policy source outlets across 87 countries, predominantly the United States and Europe. The most frequent sources included guidelines in PubMed Central, the World Health Organization, and the Centers for Disease Control and Prevention. The authors illustrate the bridge from translational research to public policy with case studies of 6 articles based on CTSA-supported research and having notable policy impact. They found articles with greater clinical relevance, altmetric attention (i.e., nonacademic community/public attention), and academic citation influence were more likely to be referenced in policy literature. Study findings help to characterize the kinds of research that have influenced and may be expected to influence health policy in the future.

[1]  Martin Szomszor,et al.  Overton: A bibliometric database of policy document citations , 2022, Quantitative Science Studies.

[2]  K. Emmons,et al.  Embracing policy implementation science to ensure translation of evidence to cancer control policy. , 2021, Translational behavioral medicine.

[3]  T. Kuchenmüller,et al.  Translating health information into policy-making: A pragmatic framework. , 2021, Health policy.

[4]  Christopher P. Austin Opportunities and challenges in translational science , 2021, Clinical and translational science.

[5]  Étienne Vignola-Gagné,et al.  A large-scale validation of the relationship between cross-disciplinary research and its uptake in policy-related documents, using the novel Overton altmetrics database , 2021, Quantitative Science Studies.

[6]  S. Woolf,et al.  Excess Deaths From COVID-19 and Other Causes in the US, March 1, 2020, to January 2, 2021. , 2021, JAMA.

[7]  A. Fitzpatrick,et al.  Big splashes & ripple effects: a narrative review of the short- & long-term impact of publications supported by an NIH CTSA pediatrics program , 2021, Translational pediatrics.

[8]  Kalene Morozumi,et al.  Evaluating a CTSA-funded pilot grant program , 2020, Journal of Clinical and Translational Science.

[9]  Dorothy R Carter,et al.  Using the payback framework to evaluate the outcomes of pilot projects supported by the Georgia Clinical and Translational Science Alliance , 2020, Journal of Clinical and Translational Science.

[10]  Dorothy R. Carter,et al.  Scope, Influence, and Interdisciplinary Collaboration: The Publication Portfolio of the NIH Clinical and Translational Science Awards (CTSA) Program From 2006 Through 2017 , 2020, Evaluation & the health professions.

[11]  J. Mascola,et al.  An mRNA Vaccine against SARS-CoV-2 — Preliminary Report , 2020, The New England journal of medicine.

[12]  Roy T Sabo,et al.  Excess Deaths From COVID-19 and Other Causes, March-April 2020. , 2020, JAMA.

[13]  J. Buse,et al.  Bibliometrics approach to evaluating the research impact of CTSAs: A pilot study , 2020, Journal of Clinical and Translational Science.

[14]  S. Woolf,et al.  Life Expectancy and Mortality Rates in the United States, 1959-2017. , 2019, JAMA.

[15]  Sidney C. Smith,et al.  2019 AHA/ACC Clinical Performance and Quality Measures for Adults With High Blood Pressure: A Report of the American College of Cardiology/American Heart Association Task Force on Performance Measures. , 2019, Circulation. Cardiovascular quality and outcomes.

[16]  George M. Santangelo,et al.  Predicting translational progress in biomedical research , 2019, PLoS biology.

[17]  Christopher P. Austin,et al.  The Fundamental Characteristics of a Translational Scientist. , 2019, ACS pharmacology & translational science.

[18]  Dorothy R. Carter,et al.  Charting the Publication and Citation Impact of the NIH Clinical and Translational Science Awards (CTSA) Program From 2006 Through 2016 , 2018, Academic medicine : journal of the Association of American Medical Colleges.

[19]  S. Elmore,et al.  The Altmetric Attention Score: What Does It Mean and Why Should I Care? , 2018, Toxicologic pathology.

[20]  C. Escoffery,et al.  Building diverse careers in clinical and translational research: Evaluation of a certificate program in translational research , 2018, Journal of Clinical and Translational Science.

[21]  Harvey A. Whiteford,et al.  How do we define the policy impact of public health research? A systematic review , 2017, Health Research Policy and Systems.

[22]  Amy M. Suiter,et al.  The Translational Science Benefits Model: A New Framework for Assessing the Health and Societal Benefits of Clinical and Translational Sciences , 2017, Clinical and translational science.

[23]  Ariela M. Freedman,et al.  Improving clinical and translational research training: a qualitative evaluation of the Atlanta Clinical and Translational Science Institute KL2-mentored research scholars program , 2016, Journal of Investigative Medicine.

[24]  George M. Santangelo,et al.  Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level , 2015, bioRxiv.

[25]  Bill Bynum,et al.  Lancet , 2015, The Lancet.

[26]  Bernadette A. Thomas,et al.  Global, regional, and national age–sex specific all-cause and cause-specific mortality for 240 causes of death, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013 , 2015, The Lancet.

[27]  B. Stoll,et al.  Maternal Influenza Immunization and Reduced Likelihood of Prematurity and Small for Gestational Age Births: A Retrospective Cohort Study , 2011, PLoS medicine.

[28]  Robert M Califf,et al.  Linking Scientific Discovery and Better Health for the Nation: The First Three Years of the NIH's Clinical and Translational Science Awards , 2010, Academic medicine : journal of the Association of American Medical Colleges.

[29]  Dwight J Rouse,et al.  A multicenter, randomized trial of treatment for mild gestational diabetes. , 2009, The New England journal of medicine.

[30]  Miguel Angel Martin-Piedra,et al.  Methods for identifying biomedical translation: a systematic review. , 2022, American journal of translational research.

[31]  C. Escobar Cervantes,et al.  [A randomized trial of intensive versus standard blood pressure control]. , 2016, Semergen.

[32]  Euan Adie,et al.  The rise of altmetrics , 2015 .

[33]  Mary Lyn Gaffield,et al.  Grading of Recommendations Assessment, Development, and Evaluation (GRADE) , 2011 .