Is research funding always beneficial? A cross-disciplinary analysis of U.K. research 2014–20

Whilst funding is essential for some types of research and beneficial for others, it may constrain academic choice and creativity. Thus, it is important to check whether it ever seems unnecessary. Here we investigate whether funded UK research tends to be higher quality in all fields and for all major research funders. Based on peer review quality scores for 113,877 articles from all fields in the UK’s Research Excellence Framework (REF) 2021, we estimate that there are substantial disciplinary differences in the proportion of funded journal articles, from Theology and Religious Studies (16%+) to Biological Sciences (91%+). The results suggest that funded research is likely to be higher quality overall, for all the largest research funders, and for 30 out of 34 REF Units of Assessment (disciplines or sets of disciplines), even after factoring out research team size. There are differences between funders in the average quality of the research supported, however. Funding seems particularly associated with higher research quality in health-related fields. The results do not show cause and effect and do not take into account the amount of funding received but are consistent with funding either improving research quality or being won by high quality researchers or projects. https://www.webofscience.com/api/gateway/wos/peer-review/10.1162/qss_a_00254

[1]  D. Ginther,et al.  Gender and Underrepresented Minority Differences in Research Funding , 2022, SSRN Electronic Journal.

[2]  Peter Edlund,et al.  Stress-Inducing and Anxiety-Ridden: A Practice-Based Approach to the Construction of Status-Bestowing Evaluations in Research Funding , 2022, Minerva.

[3]  S. Tellmann The Societal Territory of Academic Disciplines: How Disciplines Matter to Society , 2022, Minerva.

[4]  M. Cattaneo,et al.  Conditions that do or do not disadvantage interdisciplinary research proposals in project evaluation , 2022, J. Assoc. Inf. Sci. Technol..

[5]  K. Hussinger,et al.  The long-term effect of research grants on the scientific output of university professors , 2021, SSRN Electronic Journal.

[6]  Hanna Hottenrott,et al.  The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants , 2021, Humanities and Social Sciences Communications.

[7]  Simona Carini,et al.  Recent Academic Research on Clinically Relevant Digital Measures: Systematic Review. , 2021, Journal of medical Internet research.

[8]  Saeed Roshani,et al.  What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplines , 2021, Scientometrics.

[9]  P. Mongeon,et al.  Getting to the bottom of research funding: Acknowledging the complexity of funding dynamics , 2020, PloS one.

[10]  Emilio Delgado López-Cózar,et al.  Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations , 2020, Scientometrics.

[11]  Jing Qi,et al.  Performance evaluation of climate-adaptive natural ventilation design: A case study of semi-open public cultural building , 2020, Indoor and Built Environment.

[12]  A. Farr,et al.  Management of iron deficiency anaemia in secondary care across England between 2012 and 2018: a real-world analysis of Hospital Episode Statistics , 2020, Frontline Gastroenterology.

[13]  Xianwen Wang,et al.  The effect of competitive public funding on scientific output: A comparison between China and the EU , 2020, Research Evaluation.

[14]  R. Edwards Why do academics do unfunded research? Resistance, compliance and identity in the UK neo-liberal university , 2020, Studies in Higher Education.

[15]  T. Kwee,et al.  Funding of Radiology Research: Frequency and Association With Citation Rate. , 2020, AJR. American journal of roentgenology.

[16]  István Szabó,et al.  Research funding: past performance is a stronger predictor of future scientific output than reviewer scores , 2020, J. Informetrics.

[17]  S. Naveed,et al.  Psychological burden of quarantine in children and adolescents: A rapid systematic review and proposed solutions , 2020, Pakistan journal of medical sciences.

[18]  J. Levy,et al.  Quantifying the impact of housing interventions on indoor air quality and energy consumption using coupled simulation models , 2020, Journal of Exposure Science & Environmental Epidemiology.

[19]  Mike Thelwall,et al.  Academic collaboration rates and citation associations vary substantially between countries and fields , 2019, J. Assoc. Inf. Sci. Technol..

[20]  João M. Santos,et al.  Organisational factors and academic research agendas: an analysis of academics in the social sciences , 2020, Studies in Higher Education.

[21]  June Jeon Invisibilizing politics: Accepting and legitimating ignorance in environmental sciences , 2019, Social studies of science.

[22]  S. Sörlin,et al.  Co-existing Notions of Research Quality: A Framework to Study Context-specific Understandings of Good Research , 2019, Minerva.

[23]  N. Elliott,et al.  Evidence-based practice among epilepsy specialist nurses in the Republic of Ireland: Findings from the SENsE study. , 2019, Journal of nursing management.

[24]  Adrián A. Díaz-Faes,et al.  What characterises funded biomedical research? Evidence from a basic and a clinical domain , 2019, Scientometrics.

[25]  Japanese experience with centres of excellence (Promoting Research Excellence: New Approaches to Funding) , 2019 .

[26]  Dar-Zen Chen,et al.  Tracking research performance before and after receiving the Cheung Kong Scholars award: A case study of recipients in 2005 , 2018, Research Evaluation.

[27]  R. Saygitov The Impact of Grant Funding on the Publication Activity of Awarded Applicants: A Systematic Review of Comparative Studies and Meta-analytical Estimates , 2018, bioRxiv.

[28]  M. Fenton,et al.  Outcomes of early NIH-funded investigators: Experience of the National Institute of Allergy and Infectious Diseases , 2018, bioRxiv.

[29]  A. Nyamapfene Teaching-only Academics in a Research Intensive University: From an undesirable to a desirable academic identity , 2018 .

[30]  A. Lyndon Influence of the FSBI Small Research Grants scheme: an analysis and appraisal. , 2018, Journal of fish biology.

[31]  S. de Rijcke,et al.  The Drawbacks of Project Funding for Epistemic Innovation: Comparing Institutional Affordances and Constraints of Different Types of Research Funding , 2018, Minerva.

[32]  Deepak L. Bhatt,et al.  Availability and Use of Shared Data From Cardiometabolic Clinical Trials , 2017, Circulation.

[33]  J. Kim,et al.  Quality of randomized controlled trials published in the International Urogynecology Journal 2007–2016 , 2018, International Urogynecology Journal.

[34]  Min Song,et al.  The funding factor: a cross-disciplinary examination of the association between research funding and citation impact , 2018, Scientometrics.

[35]  Adrián A. Díaz-Faes,et al.  Beyond funding: Acknowledgement patterns in biomedical, natural and social sciences , 2017, PloS one.

[36]  T. Winzenberg,et al.  Growing and retaining general practice research leaders in Australia: How can we do better? , 2017, Australian family physician.

[37]  J. Freedman,et al.  Peer Review Practices for Evaluating Biomedical Research Grants: A Scientific Statement From the American Heart Association. , 2017, Circulation research.

[38]  Michael Levitt,et al.  Future of fundamental discovery in US biomedical research , 2017, Proceedings of the National Academy of Sciences.

[39]  Mike Thelwall,et al.  The accuracy of confidence intervals for field normalised indicators , 2017, J. Informetrics.

[40]  Hanna Hottenrott,et al.  Fishing for Complementarities: Research Grants and Research Productivity , 2017 .

[41]  Sonia M. Thomas,et al.  Strategies to Prevent or Reduce Gender Bias in Peer Review of Research Grants: A Rapid Scoping Review , 2017, PloS one.

[42]  Naotoshi Tsukada,et al.  Promoting diversity in science in Japan through mission-oriented research grants , 2017, Scientometrics.

[43]  Mike Thelwall,et al.  Three practical field normalised alternative indicator formulae for research evaluation , 2016, J. Informetrics.

[44]  Jun Suzuki,et al.  Promoting diversity in science in Japan through mission-oriented research grants , 2017, Scientometrics.

[45]  Fulvio Castellacci,et al.  Who gets Horizon 2020 research grants? Propensity to apply and probability to succeed in a two-step analysis , 2016, Scientometrics.

[46]  Alessandro Muscio,et al.  The complex relationship between academic engagement and research output: Evidence from Italy , 2016 .

[47]  F. Iqbal,et al.  A Bibliometric Analysis of the 100 Most-cited Articles in Rhinoplasty , 2016, Plastic and reconstructive surgery. Global open.

[48]  Alina Martinez,et al.  Developing internationally engaged scientists and engineers: The effectiveness of an international postdoctoral fellowship program , 2016 .

[49]  S. Emery,et al.  Fracture of fusion mass after hardware removal in patients with high sagittal imbalance. , 2016, Journal of neurosurgery. Spine.

[50]  Vincent Larivière,et al.  Concentration of research funding leads to decreasing marginal returns , 2016, ArXiv.

[51]  A. Casadevall,et al.  NIH peer review percentile scores are poorly predictive of grant productivity , 2016, eLife.

[52]  Thomas Sinkjær,et al.  Size, Accumulation and Performance for Research Grants: Examining the Role of Size for Centres of Excellence , 2016, PloS one.

[53]  J. Neufeld Determining effects of individual research grants on publication output and impact: The case of the Emmy Noether Programme (German Research Foundation) , 2016 .

[54]  Pedro Antonio Gutiérrez,et al.  Ordinal Regression Methods: Survey and Experimental Study , 2016, IEEE Transactions on Knowledge and Data Engineering.

[55]  Andrea Schiffauerova,et al.  How to boost scientific production? A statistical analysis of research funding and other influencing factors , 2016, Scientometrics.

[56]  Mike Thelwall,et al.  Alternative metric indicators for funding scheme evaluations , 2016, Aslib J. Inf. Manag..

[57]  R. Buchbinder,et al.  The effect of journal impact factor, reporting conflicts, and reporting funding sources, on standardized effect sizes in back pain trials: a systematic review and meta-regression , 2015, BMC Musculoskeletal Disorders.

[58]  B. Currie-Alder Research for the Developing World: Public Funding from Australia, Canada, and the UK , 2015 .

[59]  A. Jaffe,et al.  The effect of public funding on research output: the New Zealand Marsden Fund , 2015 .

[60]  G. Sivertsen,et al.  Options and limitations in measuring the impact of research grants—evidence from Denmark and Norway , 2015 .

[61]  M. D. Lindner,et al.  Examining the Predictive Validity of NIH Peer Review Scores , 2015, PloS one.

[62]  C. Bloch,et al.  The size of research funding: Trends and implications , 2015 .

[63]  B. Kehm,et al.  Academic Careers and Work in Europe: Trends, Challenges, Perspectives , 2015 .

[64]  Gelin Xu,et al.  NSFC Health Research Funding and Burden of Disease in China , 2014, PloS one.

[65]  Thed N. van Leeuwen,et al.  Analysing robustness and uncertainty levels of bibliometric performance statistics supporting science policy. A case study evaluating Danish postdoctoral funding , 2014 .

[66]  Stephen A. Gallo,et al.  The Validation of Peer Review through Research Impact Measures and the Implications for Funding Strategies , 2014, PloS one.

[67]  Nicholas Graves,et al.  The impact of funding deadlines on personal workloads, stress and family relationships: a qualitative study of Australian researchers , 2014, BMJ Open.

[68]  S. Cro,et al.  Does source of funding and conflict of interest influence the outcome and quality of spinal research? , 2014, The spine journal : official journal of the North American Spine Society.

[69]  C. Bloch,et al.  Competitive Research Grants and Their Impact on Career Performance , 2014 .

[70]  D. Allison,et al.  Overstatement of results in the nutrition and obesity peer-reviewed literature. , 2013, American journal of preventive medicine.

[71]  Nicholas Graves,et al.  On the time spent preparing grant proposals: an observational study of Australian researchers , 2013, BMJ Open.

[72]  A. DeMaria,et al.  The magnitude and nature of unfunded published cardiovascular research. , 2011, Journal of the American College of Cardiology.

[73]  Jonathan M. Levitt Are funded articles more highly cited than unfunded articles ? A preliminary investigation , 2013 .

[74]  Abbey Hyde,et al.  The Research Function of the Academic Profession in Europe , 2013 .

[75]  G. Ateș,et al.  Academic Career Paths , 2013 .

[76]  Ulrich Teichler,et al.  The work situation of the academic profession in Europe: findings of a survey in twelve countries , 2013 .

[77]  Jian Wang,et al.  Citation time window choice for research impact evaluation , 2013, Scientometrics.

[78]  Christoph Grimpe,et al.  Extramural research grants and scientists’ funding strategies: Beggars cannot be choosers? , 2012 .

[79]  Sam Soo Kim,et al.  Characteristics and trends of radiology research: a survey of original articles published in AJR and Radiology between 2001 and 2010. , 2012, Radiology.

[80]  P. V. D. Besselaar,et al.  The Selection of Scientific Talent in the Allocation of Research Grants , 2012 .

[81]  Hanna Hottenrott,et al.  Research grants, sources of ideas and the effects on academic research , 2012 .

[82]  Svein Kyvik,et al.  Academic work from a comparative perspective: a survey of faculty working time across 13 countries , 2012 .

[83]  D. Hicks Performance-based university research funding systems , 2012 .

[84]  J. Rigby Systematic grant and funding body acknowledgement data for publications: new dimensions and new controversies for research policy and evaluation , 2011 .

[85]  Ali Gazni,et al.  The effect of funding on academic research impact: a case study of Iranian publications , 2011, Aslib Proc..

[86]  D. Cox,et al.  Considerations, clues and challenges: Gaining Ethical and Trust research approval when using the NHS as a research setting , 2011 .

[87]  Joan B. Hirt,et al.  Reshaping academic capitalism to meet development priorities: the case of public universities in Kenya , 2011 .

[88]  B. Thyer Harmful Effects of Federal Research Grants , 2011 .

[89]  Hanna Hottenrott,et al.  Industry Funding of University Research and Scientific Productivity , 2011 .

[90]  Gwen Ottinger,et al.  Undone Science: Charting Social Movement and Civil Society Challenges to Research Agenda Setting , 2010, Science, technology & human values.

[91]  A. Metcalfe Revisiting Academic Capitalism in Canada: No Longer the Exception , 2010 .

[92]  J. Ioannidis,et al.  Sources of funding for Nobel Prize‐winning work: public or private? , 2010, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[93]  A. Olejniczak,et al.  The Effects of Scholarly Productivity and Institutional Characteristics on the Distribution of Federal Research Grants , 2010 .

[94]  Dangzhi Zhao,et al.  Characteristics and impact of grant-funded research: a case study of the library and information science field , 2010, Scientometrics.

[95]  L. Gruppen,et al.  A Small Grants Program Improves Medical Education Research Productivity , 2009, Academic medicine : journal of the Association of American Medical Colleges.

[96]  L. Leydesdorff,et al.  Past performance, peer review and project selection: a case study in the social and behavioral sciences , 2009, 0911.1306.

[97]  Ulf Sandström,et al.  Research quality and diversity of funding: A model for relating research money to output of research , 2009, Scientometrics.

[98]  M. Wright,et al.  Funding incentives, collaborative dynamics and scientific productivity : Evidence from the EU framework program , 2009 .

[99]  D. Skaggs,et al.  Is Research Presented at the Scoliosis Research Society Annual Meeting Influenced by Industry Funding? , 2008, Spine.

[100]  Hugo Horta,et al.  Does competitive research funding encourage diversity in higher education , 2008 .

[101]  Daniel Chudnovsky,et al.  Money for Science? The Impact of Research Grants on Academic Output* , 2008 .

[102]  Brent Goldfarb,et al.  The effect of government contracting on academic research: Does the source of funding affect scientific output , 2008 .

[103]  P. V. D. Besselaar,et al.  Comparing the evolution of national research policies: what patterns of change? , 2007 .

[104]  Claire Polster The nature and implications of the growing importance of research grants to Canadian universities and academics , 2007 .

[105]  Gerlese S. Åkerlind,et al.  Autonomy in Social Science Research: The View from United Kingdom and Australian Universities , 2007 .

[106]  Julianne Cheek,et al.  THE PRACTICE AND POLITICS OF FUNDED QUALITATIVE RESEARCH , 2007 .

[107]  Lutz Bornmann,et al.  Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants , 2006, Scientometrics.

[108]  Grant Lewison,et al.  The effect of funding on the outputs of biomedical research , 2006, Scientometrics.

[109]  G. Laudel Is external research funding a valid indicator for research performance , 2005 .

[110]  Bluma C. Peritz,et al.  The citation impact of funded and unfunded research in economics , 1990, Scientometrics.

[111]  Thomas Banchoff,et al.  Institutions, Inertia and European Union Research Policy , 2002 .

[112]  A. J. Johnson,et al.  American Society of Neuroradiology research survey 2001. , 2002, Academic radiology.

[113]  Monica Gaughan,et al.  Using curriculum vitae to compare some impacts of NSF research grants with research center funding , 2002 .

[114]  Philippe Larédo,et al.  Innovation and research policy in France (1980-2000) or the disappearance of the Colbertist state , 2002 .

[115]  O Jolobe,et al.  Guidelines for the management of iron deficiency anaemia , 2001, Gut.

[116]  Larry L. Leslie,et al.  Expanding and Elaborating the Concept of Academic Capitalism , 2001 .

[117]  Teresa R. Behrens,et al.  Unintended consequences of cooperative research: impact of industry sponsorship on climate for academic freedom and other graduate student outcome , 2001 .

[118]  David Demeritt,et al.  The new social contract for science: Accountability, relevance, and value in US and UK science and research policy , 2000 .

[119]  E. Silberman,et al.  Research without external funding in North American psychiatry. , 1997, The American journal of psychiatry.

[120]  S. Weiss,et al.  Research funding in the four major emergency medicine journals. , 1997, The American journal of emergency medicine.

[121]  Jennifer Platt Has Funding Made a Difference to Research Methods? , 1996 .

[122]  J. Bodensteiner The Saga of the Septum Pellucidum: A Tale of Unfunded Clinical Investigations , 1995, Journal of child neurology.

[123]  J. Berman,et al.  Impact of Unfunded Research in Medicine, Pathology, and Surgery , 1995, Southern medical journal.

[124]  Daryl E. Chubin,et al.  Grants Peer Review in Theory and Practice , 1994 .

[125]  M. Stein,et al.  Who pays for published research? , 1993, JAMA.

[126]  J. Berman,et al.  Research by pathologists not funded by external grant agencies: a success story. , 1992, Modern pathology : an official journal of the United States and Canadian Academy of Pathology, Inc.

[127]  M. Crosland,et al.  The Emergence of Research Grants within the Prize System of the French Academy of Sciences, 1795-1914 , 1989 .

[128]  R. Whitley The Intellectual and Social Organization of the Sciences (Second Edition: with new introductory chapter entitled 'Science Transformed? The Changing Nature of Knowledge Production at the End of the Twentieth Century') , 1984 .