What happens when national research funding is linked to differentiated publication counts? A comparison of the Australian and Norwegian publication-based funding models

The experiences from Australia where undifferentiated publication counts were linked to funding of universities in 1993 is well known. Publication activity increased, but the largest increase was in lower-impact journals, leading to a general drop in overall citation impact for Australia. The experience from Australia has been a warning for what would most likely happen if funding were linked to publication activity. Nevertheless, in 2005, a performance-based model based on differentiated publication counts was implemented in Norway. The model was specifically developed to counter adverse effects like those identified in the Australian case. In the present article, we examine ‘what happens at the aggregated level of publication and citation activity when funding is linked to differentiated publication counts’. We examine developments in Norwegian publication activity, journal publication profiles, and citation impact. We also examine developments in publication activities at the individual level and developments in research and development resource inputs. We compare experiences in Australia to those in Norway. The results show that for the Norwegian case, overall publication activity goes up, impact remains stable, and there is no indication of a deliberate displacement of journal publication activities to the lowest-impact journals. Hence, we do not see the same patterns as in Australia. We conclude that the experience in Norway with differentiated publication counts linked to funding has been different from the experience in Australia with an undifferentiated model. This is an important observation because currently the Norwegian model is being or has been adopted in several European countries.

[1]  L. Butler.  A list of published papers is no measure of value , 2002, Nature.

[2]  L. Butler,et al.  Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts , 2003 .

[3]  Thed N. van Leeuwen,et al.  The Leiden ranking 2011/2012: Data collection, indicators, and interpretation , 2012, J. Assoc. Inf. Sci. Technol..

[4]  Linda Butler,et al.  Impacts of performance-based research funding systems: A review of the concerns and the evidence , 2010 .

[5]  Jesper W. Schneider.  Bibliometric Research Performance Indicators for the Nordic Countries : A publication from the NORIA-net "The use of bibliometrics in research policy and evaluation activities" , 2010 .

[6]  Kaare Aagaard,et al.  How incentives trickle down: Local use of a national bibliometric indicator system , 2015 .

[7]  Richard A. Berk,et al.  What You Can and Can’t Properly Do with Regression , 2010 .

[8]  Edward E. Leamer,et al.  Let's Take the Con Out of Econometrics , 1983 .

[9]  L. Butler,et al.  Modifying publication practices in response to funding formulas , 2003 .

[10]  J. Schneider.  An Outline of the Bibliometric Indicator Used for Performance-Based Funding of Research Institutions in Norway , 2009 .

[11]  Linda Butler,et al.  What Happens when Funding is Linked to Publication Counts , 2004 .

[12]  D. Hicks.  Performance-based university research funding systems , 2012 .

[13]  R. Berk.  Regression Analysis: A Constructive Critique , 2003 .

[14]  J. Schneider,et al.  Developing a methodology to assess the impact of research grant funding: a mixed methods approach. , 2014, Evaluation and program planning.

[15]  D. Freedman.  Statistical models and shoe leather , 1989 .

[16]  Kaare Aagaard,et al.  Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator , 2015 .

[17]  Emilio Delgado López-Cózar,et al.  The evolution of research activity in Spain , 2003, Research Policy.

[18]  Stanley Lieberson,et al.  Making It Count: The Improvement of Social Research and Theory. , 1987 .

[19]  Laura Cruz-Castro,et al.  Overturning some assumptions about the effects of evaluation systems on publication performance , 2011, Scientometrics.