Does massive funding support of researchers work?: Evaluating the impact of the South African research chair funding initiative

Abstract In this study we evaluate whether a substantial increase in public funding to researchers is associated with a material difference in their productivity. We compare performance measures of researchers who were granted substantial funding against researchers with similar scholarly standing who did not receive such funding. We find that substantial funding is associated with raised researcher performance – though the increase is moderate, is strongly conditional on the quality of the researcher who receives the funding, and is greater in some disciplines than others. Moreover the cost per additional unit of output is such as to raise questions about the usefulness of the funding model. The implication is that public research funding will be more effective in raising research output where selectivity of recipients of funding is strongly conditional on the established track record of researchers.

[1]  Debora Shaw,et al.  A new look at evidence of scholarly citation in citation indexes and from web sources , 2008, Scientometrics.

[2]  L. Butler,et al.  Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts , 2003 .

[3]  Claire Polster A Break from the Past: Impacts and Implications of the Canada Foundation for Innovation and the Canada Research Chairs Initiatives* , 2008 .

[4]  Wolfgang Glänzel,et al.  On the Opportunities and Limitations of the H-index , 2006 .

[5]  D F Horrobin,et al.  The philosophical basis of peer review and the suppression of innovation. , 1990, JAMA.

[6]  Joel Mokyr,et al.  The Gifts of Athena: Historical Origins of the Knowledge Economy , 2002 .

[7]  Ben R. Martin,et al.  The use of multiple indicators in the assessment of basic research , 1996, Scientometrics.

[8]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007 .

[9]  Sebastian Spaeth,et al.  How constraints and knowledge impact open innovation , 2013 .

[10]  P. Howitt,et al.  Testing Creative Destruction in an Opening Economy , 2013 .

[11]  James Testa,et al.  The Thomson Scientific journal selection process. , 2006, International microbiology : the official journal of the Spanish Society for Microbiology.

[12]  Péter Jacsó,et al.  Metadata mega mess in Google Scholar , 2010, Online Inf. Rev..

[13]  Jacques Mairesse,et al.  Research Investment, Innovation and Productivity: An Econometric Analysis at the Firm Level , 1998 .

[14]  Juan E. Iglesias,et al.  Scaling the h-index for different scientific ISI fields , 2006, Scientometrics.

[15]  Thomas E. Nisonger,et al.  Citation Autobiography: An Investigation of ISI Database Coverage in Determining Author Citedness , 2004 .

[16]  A. Salter,et al.  Open for innovation: the role of openness in explaining innovation performance among U.K. manufacturing firms , 2006 .

[17]  N. Mclaughlin,et al.  The Canada Research Chairs Program and Social Science Reward Structures , 2008 .

[18]  Constance E. Helfat,et al.  INNOVATION OBJECTIVES, KNOWLEDGE SOURCES, AND THE BENEFITS OF BREADTH , 2010 .

[19]  Lutz Bornmann,et al.  Does the h-index for ranking of scientists really work? , 2005, Scientometrics.

[20]  Anne-Wil Harzing,et al.  Google Scholar as a new source for citation analysis , 2008 .

[21]  Lutz Bornmann,et al.  What do we know about the h index? , 2007, J. Assoc. Inf. Sci. Technol..

[22]  Lokman I. Meho,et al.  A New Era in Citation and Bibliometric Analyses: Web of Science, Scopus, and Google Scholar , 2006, ArXiv.

[23]  D. Mowery,et al.  Technology and the pursuit of economic growth , 1991 .

[24]  J. W. Fedderke The objectivity of national research foundation peer review in South Africa assessed against bibliometric indexes , 2013, Scientometrics.

[25]  Mike Thelwall,et al.  Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines , 2008, Scientometrics.

[26]  M. Thelwall,et al.  Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis , 2007 .

[27]  Ed J. Rinia,et al.  COMPARATIVE ANALYSIS OF A SET OF BIBLIOMETRIC INDICATORS AND CENTRAL PEER REVIEW CRITERIA. EVALUATION OF CONDENSED MATTER PHYSICS IN THE NETHERLANDS , 1998 .

[28]  Andreas Thor,et al.  Convergent validity of bibliometric Google Scholar data in the field of chemistry - Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts , 2009, J. Informetrics.

[29]  Péter Jacsó,et al.  Dubious hit counts and cuckoo's eggs , 2006, Online Inf. Rev..

[30]  Gustavo Manso,et al.  Is Pay-for-Performance Detrimental to Innovation? , 2012 .

[31]  Reinhilde Veugelers,et al.  In Search of Complementarity in Innovation Strategy: Internal R&D and External Knowledge Acquisition , 2006, Manag. Sci..

[32]  L. Bornmann,et al.  Reference standards and reference multipliers for the comparison of the citation impact of papers published in different time periods , 2010 .

[33]  J. C. Korevaar,et al.  Validation of bibliometric indicators in the field of mathematics , 1996, Scientometrics.

[34]  Leo Egghe,et al.  An informetric model for the Hirsch-index , 2006, Scientometrics.

[35]  Lutz Bornmann,et al.  Scientific peer review , 2011, Annu. Rev. Inf. Sci. Technol..

[36]  A. Kulkarni,et al.  Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. , 2009, JAMA.

[37]  Constance E. Helfat,et al.  Location, Decentralization, and Knowledge Sources for Innovation , 2011, Organ. Sci..

[38]  D. Hicks Performance-based university research funding systems , 2012 .

[39]  Jeroen Bosman,et al.  Scopus reviewed and compared: the coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar , 2006 .

[40]  Haiyang Li,et al.  Product Innovation Strategy and the Performance of New Technology Ventures in China , 2001 .

[41]  Richard R. Nelson,et al.  The Rise and Fall of American Technological Leadership: The Postwar Era in Historical Perspective , 1992 .

[42]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[43]  Jacques Mairesse,et al.  Accounting for Innovation and Measuring Innovativeness: An Illustrative Framework and an Application , 2002 .

[44]  Christoph Grimpe,et al.  Extramural research grants and scientists’ funding strategies: Beggars cannot be choosers? , 2012 .

[45]  Péter Jacsó,et al.  Deflated, inflated and phantom citation counts , 2006, Online Inf. Rev..

[46]  Matthew E Falagas,et al.  Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses , 2007, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[47]  Anthony F. J. van Raan,et al.  Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods , 2005, Scientometrics.

[48]  Bruno Crépon,et al.  Research, Innovation, and Productivity: an Econometric Analysis at the Firm Level , 1998 .

[49]  Yang Li-ying,et al.  Bibliometrical Analysis of Paper's Co-occurrence in Main International Institutions&mdash|A Case Study in Chemical Research , 2006 .

[50]  J. V. Reenen Does Competition Raise Productivity Through Improving Management Quality , 2011 .

[51]  Anthony F. J. van Raan Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups , 2013, Scientometrics.

[52]  Charles Oppenheim,et al.  The correlation between citation counts and the 1992 research assessment exercise ratings for British research in genetics, anatomy and archaeology , 1997, J. Documentation.

[53]  Nathan Rosenberg,et al.  Exploring the Black Box: Technology, Economics, and History , 1994 .

[54]  Anne-Wil Harzing,et al.  REFLECTIONS ON THE H-INDEX , 2012 .

[55]  C. Rammer,et al.  Resource allocation strategy for innovation portfolio management , 2014 .

[56]  Philippe Aghion,et al.  Innovation and Institutional Ownership , 2013 .

[57]  D. Braun,et al.  The role of funding agencies in the cognitive development of science , 1998 .

[58]  Henry L. Roediger,et al.  The h Index in Science: A New Measure of Scholarly Contribution , 2006 .

[59]  Eduardo S. Schwartz,et al.  Investment Under Uncertainty. , 1994 .

[60]  W. D. Hall,et al.  A cautionary bibliometric tale of two cities , 2010, Scientometrics.

[61]  D. Aksnes,et al.  Peer reviews and bibliometric indicators: a comparative study at a Norwegian university , 2004 .

[62]  Jan Fagerberg,et al.  Technology and International Differences in Growth Rates , 1994 .

[63]  Tammy L. Madsen,et al.  Unbundling competitive heterogeneity: incentive structures and capability influences on technological innovation , 2009 .

[64]  Lokman I. Meho,et al.  Using the h-index to rank influential information scientists , 2006, J. Assoc. Inf. Sci. Technol..