A new approach for estimating research impact: An application to French cancer research

Much attention has been paid to estimating the impact of investments in scientific research. Historically, those efforts have been largely ad hoc, burdensome, and error prone. In addition, the focus has been largely mechanical—drawing a direct line between funding and outputs—rather than focusing on the scientists that do the work. Here, we provide an illustrative application of a new approach that examines the impact of research funding on individuals and their scientific output in terms of publications, citations, collaborations, and international activity, controlling for both observed and unobserved factors. We argue that full engagement between scientific funders and the research community is needed if we are to expand the data infrastructure to enable a more scientific assessment of scientific investments.

[1]  Alessandro Maffioli,et al.  The Impact of National Research Funds: A Regression Discontinuity Approach to the Chilean FONDECYT , 2012 .

[2]  Walter W. Powell,et al.  Collective Invention and Inventor Networks , 2010 .

[3]  Julia Lane,et al.  New Linked Data on Research Investments: Scientific Workforce, Productivity, and Public Value , 2014, Research policy.

[4]  Monica Gaughan,et al.  Scientific and technical human capital: an alternative model for research evaluation , 2001, Int. J. Technol. Manag..

[5]  Marco Caliendo,et al.  Some Practical Guidance for the Implementation of Propensity Score Matching , 2005, SSRN Electronic Journal.

[6]  Colin Macilwain,et al.  Science economics: What science is really worth , 2010, Nature.

[7]  Riccardo Crescenzi,et al.  Papers in Evolutionary Economic Geography # 13 . 24 Do inventors talk to strangers ? On proximity and collaborative knowledge creation , 2013 .

[8]  Adam B. Jaffe,et al.  The Changing Frontier: Rethinking Science and Innovation Policy , 2015 .

[9]  Ben R. Martin,et al.  The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster? , 2011 .

[10]  A. van de Rijt,et al.  The Matthew effect in science funding , 2018, Proceedings of the National Academy of Sciences.

[11]  P. Romer.  Endogenous Technological Change , 1989, Journal of Political Economy.

[12]  R. Scoble,et al.  Assessment, evaluations, and definitions of research impact: A review , 2014 .

[13]  W. Powell,et al.  Networks, Propinquity, and Innovation in Knowledge-intensive Industries , 2009 .

[14]  Ron S. Jarmin,et al.  Wrapping it up in a person: Examining employment and earnings outcomes for Ph.D. recipients , 2015, Science.

[15]  Pierre Azoulay.  Research efficiency: Turn the scientific method on ourselves , 2012, Nature.

[16]  P. Mohnen,et al.  The Importance of R&D and Innovation for Productivity: A Reexamination in Light of the 2000 French Innovation Survey , 2005 .

[17]  Jasjit Singh,et al.  The World is Not Small for Everyone: Inequity in Searching for Knowledge in Organizations , 2010, Manag. Sci..

[18]  J. Haskel,et al.  Knowledge Spillovers, ICT and Productivity Growth , 2014, SSRN Electronic Journal.

[19]  Philippe Larédo,et al.  ASIRPA: A comprehensive theory-based approach to assessing the societal impacts of a research organization , 2015 .

[20]  B. Bernanke.  Promoting research and development: the government's role: a speech at the Conference on \\"New Building Blocks for Jobs and Economic Growth,\\" Washington, D.C., May 16, 2011 , 2011 .

[21]  Stacy Konkiel,et al.  Dimensions: Bringing down barriers between scientometricians and data , 2020, Quantitative Science Studies.

[22]  J. Spaapen,et al.  Introducing ‘productive interactions’ in social impact assessment , 2011 .

[23]  Petra E. Todd,et al.  Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods , 2001 .

[24]  J. Robins,et al.  Doubly Robust Estimation in Missing Data and Causal Inference Models , 2005, Biometrics.

[25]  Sheila Wilson,et al.  Research Excellence Framework , 2013 .

[26]  A. Teich.  In Search of Evidence-Based Science Policy: From the Endless Frontier to Scisip , 2018 .

[27]  Javier Miranda,et al.  Measuring and Accounting for Innovation in the 21st Century , 2017 .

[28]  Using Cyber-enabled Transaction Data to Study Productivity and Innovation in Organizations , 2009 .

[29]  Raffaello Bronzini,et al.  Are Incentives for R&D Effective? Evidence from a Regression Discontinuity Approach , 2011 .

[30]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[31]  Andrea G. Trice,et al.  A PROFILE OF FEDERAL-GRANT ADMINISTRATIVE BURDEN AMONG FEDERAL DEMONSTRATION PARTNERSHIP FACULTY A Report of the Faculty Standing Committee of the Federal Demonstration Partnership , 2007 .

[32]  Susan Athey,et al.  The State of Applied Econometrics - Causality and Policy Evaluation , 2016, 1607.00699.

[33]  Juan D. Rogers,et al.  A churn model of scientific knowledge value: Internet researchers as a knowledge value collective , 2002 .

[34]  S. Breschi,et al.  Mobility of Skilled Workers and Co-Invention Networks: An Anatomy of Localized Knowledge Flows , 2009 .

[35]  Laura B. Rawlings,et al.  Impact Evaluation in Practice: Second Edition , 2010 .

[36]  John H Marburger,et al.  Wanted: Better Benchmarks , 2005, Science.

[37]  Richard Russell,et al.  Manufacturing the Future: Federal Priorities for Manufacturing Research and Development. Report of the Interagency Working Group on Manufacturing R&D, Committee on Technology, National Science and Technology Council , 2008 .

[38]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[39]  Matias D. Cattaneo,et al.  Econometric Methods for Program Evaluation , 2018, Annual Review of Economics.