Measuring changes in publication patterns in a context of performance-based research funding systems: the case of educational research in the University of Gothenburg (2005–2014)

We present a novel way to frame a discussion of changes in publication patterns that occur in a context of performance-based research funding systems (PRFSs). Adopting an approach derived from social epistemology, we foreground the dialectical nature of knowledge. This allows us to relate changes in publication patterns to PRFS and show the tensions that emerge between relatively diverse research fields with diverse publication practices and bibliometric indicators within PRFSs that reinforce a singular view of research goals. Specifically, we highlight that the employment of bibliometric indicators result in a fixed hierarchy among communication media that may be at odds with the goals within research fields subjected to PRFSs. These ideas are illustrated with an empirical analysis of changes in publication patterns within the field of educational research at the University of Gothenburg (GU; 2005–2014) in Sweden. We contrast the observed changes with implicit priorities in the national and institutional PRFS that operate in this context since 2009. Findings from bibliometric analysis indicate a move away from publication traditions that used to be characteristic of educational research: the growth in the number of journal articles is greater than that in the number of book chapters, while the number of reports is on a declining slope. In relation to PRFSs, we show that conclusive judgement on the desirability of the observed changes is hardly achievable. If one adopts the aims implicit in PRFS, research performance appears to be enhanced. If one sides with the views of many educational researchers, then some of the trends might be an indication of undesirable changes.

[1]  Jan-Erik Gustafsson,et al.  Becoming an Academic Researcher , 2014 .

[2]  David A. Tandberg,et al.  Evaluating the Impacts of “New” Performance Funding in Higher Education , 2015 .

[3]  F. Vandermoere,et al.  Scholarly Communication in AERA Journals, 1931 to 2014 , 2016 .

[4]  D. Hicks.  The Four Literatures of Social Science , 2004 .

[5]  G. Sivertsen.  Publication-Based Funding: The Norwegian Model , 2016 .

[6]  Linda Butler,et al.  What Happens when Funding is Linked to Publication Counts , 2004 .

[7]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics. , 2015, Nature.

[8]  Elisabet Öhrn,et al.  Kön och karriär i akademin : En studie inom det utbildningsvetenskapliga fältet , 2013 .

[9]  Paul Wouters,et al.  Evaluation practices and effects of indicator use : a literature review , 2016 .

[10]  D. Hicks.  Performance-based university research funding systems , 2012 .

[11]  Laura Cruz-Castro,et al.  Overturning some assumptions about the effects of evaluation systems on publication performance , 2011, Scientometrics.

[12]  Thomas Rabovsky,et al.  Using Data to Manage for Performance at Public Universities , 2014 .

[13]  Eric H. J. Spruyt,et al.  Changing publication patterns in the Social Sciences and Humanities, 2000–2009 , 2012, Scientometrics.

[14]  J. Gustafsson,et al.  Causes of educational segregation in Sweden – school choice or residential segregation , 2016 .

[15]  R. Vanderstraeten.  Scholarly Communication in Education Journals , 2011 .

[16]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics , 2015, Nature.

[17]  Thed N. van Leeuwen,et al.  Bibliometric analysis of output and impact based on CRIS data: a case study on the registered output of a Dutch university , 2015, Scientometrics.

[18]  Werner Dees,et al.  Innovative Scientometric Methods for a Continuous Monitoring of Research Activities in Educational Science , 2008 .

[19]  Kaare Aagaard,et al.  How incentives trickle down: Local use of a national bibliometric indicator system , 2015 .

[20]  Sarah de Rijcke,et al.  Accountability in context: effects of research evaluation systems on publication practices, disciplinary norms, and individual working routines in the faculty of Arts at Uppsala University , 2015 .

[21]  Tomas Englund,et al.  New Trends in Swedish Educational Research , 2006 .

[22]  Kaare Aagaard,et al.  Impacts of performance-based research funding systems: The case of the Norwegian Publication Indicator , 2015 .

[23]  Koen Jonkers,et al.  Research Performance Based Funding Systems: a Comparative Assessment , 2016 .

[24]  J. Schneider.  An Outline of the Bibliometric Indicator Used for Performance-Based Funding of Research Institutions in Norway , 2009 .

[25]  H. Moed,et al.  The use of bibliometric data for the measurement of university research performance , 1985 .

[26]  Dejan Pajic,et al.  Globalization of the social sciences in Eastern Europe: genuine breakthrough or a slippery slope of the research evaluation practice? , 2015, Scientometrics.

[27]  L. Butler,et al.  Modifying publication practices in response to funding formulas , 2003 .

[28]  Kevin J. Dougherty,et al.  Performance Funding for Higher Education , 2014 .

[29]  Jochen Gläser,et al.  A fight on epistemological quicksand: Comment on the dispute between van den Besselaar et al. and Butler , 2017, J. Informetrics.

[30]  Emilio Delgado López-Cózar,et al.  The evolution of research activity in Spain , 2003, Research Policy.

[31]  Henk F. Moed,et al.  UK Research Assessment Exercises: Informed judgments on research quality or quantity? , 2008, Scientometrics.

[32]  Kaare Aagaard,et al.  What happens when national research funding is linked to differentiated publication counts? A comparison of the Australian and Norwegian publication-based funding models , 2016 .

[33]  Birger Larsen,et al.  Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: an empirical analysis of the potential , 2011, Scientometrics.