The impact of research funding on scientific outputs: Evidence from six smaller European countries

We investigate the relationships between the citation impacts of scientific papers and the sources of funding that are acknowledged as having supported those publications. We examine several relationships potentially associated with funding, including first citation, total citations, and the chances of becoming highly cited. Furthermore, we explore the links between citations and types of funding by organization and also with combined measures of funding. In particular, we examine the relationship between funding intensity and funding variety and citation. Our empirical work focuses on six small advanced European economies, applying a zero inflated negative binomial model to a set of more than 240,000 papers authored by researchers from these countries. We find that funding is not related to the first citation but is significantly related to the number of citations and top percentile citation impact. Additionally, we find that citation impact is positively related to funding variety and negatively related with funding intensity. Finally there is an inverse relationship between the relative frequency of funding and citation impact. The results presented in the paper provide insights for the design of research programs and the structure of research funding and for the behavior and strategies of research and sponsoring organizations.

[1]  J. Rigby.  Systematic grant and funding body acknowledgement data for publications: new dimensions and new controversies for research policy and evaluation , 2011 .

[2]  Mike Thelwall,et al.  Determinants of research citation impact in nanoscience and nanotechnology , 2013, J. Assoc. Inf. Sci. Technol..

[3]  John Rigby,et al.  Optimizing research impact by allocating funding to researcher grant portfolios: Some evidence on a policy option (rip) , 2013 .

[4]  Bruce A. Desmarais,et al.  Testing for Zero Inflation in Count Models: Bias Correction for the Vuong Test , 2013 .

[5]  J. A. Calvin.  Regression Models for Categorical and Limited Dependent Variables , 1998 .

[6]  Ammon Salter,et al.  The impact of entrepreneurial capacity, experience and organizational support on academic entrepreneurship , 2011 .

[7]  D. Mowery,et al.  Technology policy and global warming: Why new policy models are needed (or why putting new wine in old bottles won't work) , 2010 .

[8]  Thed N. van Leeuwen,et al.  Approaching the "reward triangle": General analysis of the presence of funding acknowledgments and "peer interactive communication" in scientific publications , 2012, J. Assoc. Inf. Sci. Technol..

[9]  Grant Lewison,et al.  The effect of funding on the outputs of biomedical research , 2006, Scientometrics.

[10]  R. Whitley.  The Intellectual and Social Organization of the Sciences (Second Edition: with new introductory chapter entitled 'Science Transformed? The Changing Nature of Knowledge Production at the End of the Twentieth Century') , 1984 .

[11]  Robin Teigland,et al.  Evaluating a Research Funding Program: Measuring the Impact of EU 6 Framework Programme , 2009 .

[12]  Ismael Rafols,et al.  Is science becoming more interdisciplinary? Measuring and mapping six research fields over time , 2009, Scientometrics.

[13]  William J. Sutherland,et al.  How effective are European agri‐environment schemes in conserving and promoting biodiversity? , 2003 .

[14]  Fuyuki Yoshikane.  Multiple regression analysis of a patent’s citation frequency and quantitative characteristics: the case of Japanese patents , 2013, Scientometrics.

[15]  Ulf Sandström,et al.  The effects of funding modes on the quality of knowledge production , 2013 .

[16]  Philip Shapira,et al.  Funding acknowledgement analysis: an enhanced tool to investigate research sponsorship impacts: the case of nanotechnology , 2011, Scientometrics.

[17]  Manuel Acosta,et al.  Factors affecting inter-regional academic scientific collaboration within Europe: the role of economic distance , 2011, Scientometrics.

[18]  John T. Scott,et al.  The Nature of Innovation Market Failure and the Design of Public Support for Private Innovation , 2000 .

[19]  G. Nemet.  Demand-pull, technology-push, and government-led incentives for non-incremental technical change , 2009 .

[20]  Michael Jennings.  Academic output from EU-funded health research projects , 2012, The Lancet.

[21]  J. Arnold,et al.  SWAT2000: current capabilities and research opportunities in applied watershed modelling , 2005 .

[22]  Cristina Bayona-Sáez,et al.  Assessing the effectiveness of the Eureka Program , 2010 .

[23]  Grant Lewison,et al.  Beyond outputs: new measures of biomedical research impact , 2003, Aslib Proc..

[24]  L Leiserson,et al.  Project hindsight. , 1967, Science.

[25]  Philip Shapira,et al.  Is There a Relationship between Research Sponsorship and Publication Impact? An Analysis of Funding Acknowledgments in Nanotechnology Papers , 2015, PloS one.

[26]  Magnus Hagevi,et al.  Follow the money: Public subsidies and the changing intra-party balance of power between different faces of the party organisation—the case of Sweden , 2018 .

[27]  Luke Georghiou,et al.  Evaluation of the impact of European Community research programmes upon the competitiveness of European industry - Concepts and approaches. EUR 14198 EN. Research evaluation. Science and Technology Policy Series , 1992 .

[28]  S. Schwartzman,et al.  The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies , 1994 .

[29]  Philippe Larédo,et al.  The networks promoted by the framework programme and the questions they raise about its formulation and implementation , 1998 .

[30]  Erik Arnold,et al.  Understanding long-term impacts of R&D funding: The EU framework programme , 2012 .

[31]  Jeong-Dong Lee,et al.  An in-depth empirical analysis of patent citation counts using zero-inflated count data model: The case of KIST , 2007, Scientometrics.

[32]  Xianwen Wang,et al.  Science funding and research output: a study on 10 countries , 2011, Scientometrics.

[33]  Sam Wilson,et al.  What makes an article influential? Predicting impact in social and personality psychology , 2008, Scientometrics.

[34]  Andy Walker.  Perverse incentives. , 2013, Tennessee medicine : journal of the Tennessee Medical Association.

[35]  Franco Malerba,et al.  The impact of EU-funded research networks on knowledge diffusion at the regional level , 2008 .

[36]  Chaomei Chen,et al.  Predictive effects of structural variation on citation counts , 2012, J. Assoc. Inf. Sci. Technol..

[37]  Karl Kreilkamp.  Hindsight and the Real World of Science Policy , 1971 .

[38]  Lyle H. Ungar,et al.  Positioning knowledge: schools of thought and new knowledge creation , 2010, Scientometrics.

[39]  Q. Vuong.  Likelihood Ratio Tests for Model Selection and Non-Nested Hypotheses , 1989 .

[40]  P. Shapira,et al.  Organizational and institutional influences on creativity in scientific research , 2009 .

[41]  Grant Lewison,et al.  The evaluation of Russian cancer research , 2010 .

[42]  Jörg Neufeld,et al.  Peer review-based selection decisions in individual research funding, applicants’ publication strategies and performance: The case of the ERC Starting Grants , 2013 .

[43]  George Messinis,et al.  Triadic citations, country biases and patent value: the case of pharmaceuticals , 2011, Scientometrics.

[44]  Loet Leydesdorff,et al.  Universities and the global knowledge economy: A triple helix of university-industry-government relations (2nd edition) , 2001 .

[45]  S. Baldi.  Normative versus social constructivist processes in the allocation of citations : A network-analytic model , 1998 .

[46]  Alan L. Porter,et al.  Science overlay maps: A new tool for research policy and library management , 2009, J. Assoc. Inf. Sci. Technol..

[47]  W. Greene,et al.  计量经济分析 = Econometric analysis , 2009 .

[48]  John Rigby,et al.  Looking for the impact of peer review: does count of funding acknowledgements really predict research impact? , 2012, Scientometrics.

[49]  Keith Smith,et al.  Policy learning and innovation theory: an interactive and co-evolving process , 2002 .

[50]  Paul Lillrank,et al.  Design and policy choices for technology extension organizations , 2001 .

[51]  Sherwin Cw,et al.  Project hindsight. A Defense Department study of the utility of research. , 1967, Science.

[52]  G. Lewison.  Gastroenterology research in the United Kingdom: funding sources and impact , 1998, Gut.

[53]  Diana Hicks,et al.  How much is a collaboration worth? A calibrated bibliometric model , 1997, Scientometrics.

[54]  Ronald N. Kostoff,et al.  Semiquantitative methods for research impact assessment , 1993 .

[55]  Yong-Gil Lee.  Sectoral strategic differences of technological development between electronics and chemistry: a historical view from analyses of Korean-invented US patents during the period of 1989–1992 , 2009, Scientometrics.

[56]  Thomas Scherngell,et al.  Initial comparative analysis of model and peer review process for ERC starting grant proposals , 2013 .

[57]  Pablo del Río,et al.  The market failure and the systemic failure rationales in technological innovation systems , 2013 .

[58]  Terttu Luukkonen,et al.  Conservatism and risk-taking in peer review: Emerging ERC practices , 2012 .

[59]  Henry Etzkowitz,et al.  Universities and the global knowledge economy , 1997 .

[60]  J. S. Katz,et al.  What is research collaboration , 1997 .

[61]  John Rigby,et al.  Comparing the scientific quality achieved by funding instruments for single grant holders and for collaborative networks within a research system: Some observations , 2007, Scientometrics.

[62]  Grant Lewison.  Publications from the European community's biotechnology action programme (BAP): Multinationality, acknowledgement of support, and citations , 2005, Scientometrics.

[63]  Maria Nedeva,et al.  Characterizing researchers to study research funding agency impacts: The case of the European Research Council's Starting Grants , 2012 .

[64]  Paula E. Stephan,et al.  Research efficiency: Perverse incentives , 2012, Nature.

[65]  H. Grupp,et al.  Indicators for national science and technology policy: how robust are composite indicators? , 2004 .

[66]  G Lewison,et al.  International gastroenterology research: subject areas, impact, and funding. , 2001, Gut.

[67]  Pedro L. Marin,et al.  Public Policies Towards Research Joint Venture: Institutional Design and Participants’ Characteristics , 2008 .

[68]  Daniel Sirtes.  FUNDING ACKNOWLEDGEMENTS FOR THE GERMAN RESEARCH FOUNDATION (DFG). THE DIRTY DATA OF THE WEB OF SCIENCE DATABASE AND HOW TO CLEAN IT UP , 2013 .

[69]  Naoya Katayama.  Portmanteau likelihood ratio tests for model selection , 2008 .