Effects of large-scale research funding programs: a Japanese case study

This study investigates the effects of large-scale research funding from the Japanese government on the research outcomes of university researchers. To evaluate the effects, we use the difference-in-differences estimator and measure research outcomes in terms of number of papers and citation counts per paper. Our analysis shows that the funding program led to an increase in the number of papers in some fields and an increase in the citation counts in the other fields. A comparison of our estimation results with assessment data obtained from peer reviews showed important differences. Since the characteristics of research vary according to the field, bibliometrics analysis should be used along with the peer review method for a more accurate analysis of research impact.

[1]  Barry Bozeman,et al.  Academic careers, patents, and productivity: industry experience as scientific and technical human capital , 2005 .

[2]  Loet Leydesdorff,et al.  A comment to the paper by Waltman et al., Scientometrics, 87, 467–481, 2011 , 2011, Scientometrics.

[3]  Branco Ponomariov,et al.  Faculty publication productivity, collaboration, and grants velocity: using curricula vitae to compare center-affiliated and unaffiliated scientists , 2008 .

[4]  R. Rohh.  ALTERNATIVE METHODS FOR EVALUATING THE IMPACT OF INTERVENTIONS An Overview , 2001 .

[5]  Adam B. Jaffe,et al.  Reinventing Public R&D: Patent Policy and the Commercialization of National Laboratory Technologies , 2001 .

[6]  Pravin K. Trivedi,et al.  Microeconometrics: Treatment Evaluation , 2005 .

[7]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[8]  Barry Bozeman,et al.  Impacts of grants and contracts on academic researchers’ interactions with industry , 2007 .

[9]  Manuel Trajtenberg,et al.  Market Value and Patent Citations: A First Look , 2000 .

[10]  Charles Oppenheim,et al.  The Correlation between citation counts and the 1992 Research Assessment Exercise Ratings for British Library and Information Science University departments , 1995, J. Documentation.

[11]  Jeffrey M. Woodbridge.  Econometric Analysis of Cross Section and Panel Data , 2002 .

[12]  H. Theil.  Introduction to econometrics , 1978 .

[13]  Mark W. Watson.  Introduction to econometrics. , 1968 .

[14]  Jeffrey M. Wooldridge,et al.  Solutions Manual and Supplementary Materials for Econometric Analysis of Cross Section and Panel Data , 2003 .

[15]  Monica Gaughan,et al.  Using curriculum vitae to compare some impacts of NSF research grants with research center funding , 2002 .

[16]  A. Dawid.  Conditional Independence in Statistical Theory , 1979 .

[17]  Ed J. Rinia,et al.  COMPARATIVE ANALYSIS OF A SET OF BIBLIOMETRIC INDICATORS AND CENTRAL PEER REVIEW CRITERIA. EVALUATION OF CONDENSED MATTER PHYSICS IN THE NETHERLANDS , 1998 .

[18]  Charles Oppenheim,et al.  Comparing alternatives to the Web of Science for coverage of the social sciences' literature , 2007, J. Informetrics.

[19]  James J. Heckman,et al.  Longitudinal Analysis of Labor Market Data , 1985 .

[20]  A. Cameron,et al.  Microeconometrics: Methods and Applications , 2005 .

[21]  J. Heckman,et al.  Longitudinal Analysis of Labor Market Data: Alternative methods for evaluating the impact of interventions , 1985 .

[22]  L. Georghiou,et al.  Evaluation of Research: A Selection of Current Practices , 1987 .

[23]  Adam,et al.  Flows of knowledge from universities and federal laboratories: modeling the flow of patent citations over time and across institutional and geographic boundaries. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Charles Oppenheim,et al.  The correlation between citation counts and the 1992 research assessment exercise ratings for British research in genetics, anatomy and archaeology , 1997, J. Documentation.

[25]  M. E. Bustamante,et al.  A comment on this paper. , 1965 .

[26]  Mark A. Schankerman,et al.  The Quality of Ideas: Measuring Innovation with Multiple Indicators , 1999 .

[27]  Anthony F. J. van Raan.  Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups , 2013, Scientometrics.

[28]  Richard C. Anderson,et al.  Publication ratings versus peer ratings of universities , 1978, J. Am. Soc. Inf. Sci..

[29]  A. F. J. Van Raan,et al.  Influence of interdisciplinarity on peer-review and bibliometric evaluations in physics research , 2001 .

[30]  D. Harhoff,et al.  Citation Frequency and the Value of Patented Innovation , 1997 .

[31]  Myoung‐jae Lee.  Micro-Econometrics for Policy, Program, and Treatment Effects , 2005 .

[32]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[33]  Arthur J. Meadows,et al.  Citations and departmental research ratings , 2005, Scientometrics.

[34]  D. Harhoff,et al.  Citation Frequency and the Value of Patented Inventions , 1999, Review of Economics and Statistics.

[35]  Ronald N. Kostoff,et al.  Impact Assessment : State-ofthe-Art * , 1998 .

[36]  Paul Genoni,et al.  Citation analysis and peer ranking of Australian social science journals , 2010, Scientometrics.

[37]  Monica Gaughan,et al.  Using the curriculum vitae for policy research: an evaluation of National Institutes of Health center and training support on career trajectories , 2009 .

[38]  魏屹东,et al.  Scientometrics , 2018, Encyclopedia of Big Data.

[39]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007 .