Indicators as judgment devices : An empirical study of citizen bibliometrics in research evaluation

A researcher’s number of publications has been a fundamental merit in the competition for academic positions since the late 18th century. Today, the simple counting of publications has been supplem ...

[1]  Lucien Karpik.  Valuing the Unique: The Economics of Singularities , 2010 .

[2]  T. V. Leeuwen.  Testing the validity of the Hirsch-index for research assessment purposes , 2008 .

[3]  Julia Melkers,et al.  Evaluating R&D impacts : methods and practice , 1993 .

[4]  Anthony F. J. van Raan,et al.  Citation Analysis May Severely Underestimate the Impact of Clinical Research as Compared to Basic Research , 2012, PloS one.

[5]  Justus Haucap,et al.  What drives the relevance and reputation of economics journals? An update from a survey among economists , 2015, Scientometrics.

[6]  Richard Whitley,et al.  The Changing Governance of the Sciences : The Advent of Research Evaluation Systems , 2007 .

[7]  Christine Musselin,et al.  The Market for Academics , 2009 .

[8]  J. Matthews.  Trust in numbers: The pursuit of objectivity in science and public life , 1996 .

[9]  Jens Maesse,et al.  The elitism dispositif: hierarchization, discourses of excellence and organizational change in European economics , 2017 .

[10]  Vincent Larivière,et al.  Are elite journals declining? , 2013, J. Assoc. Inf. Sci. Technol..

[11]  Helvi Kyngäs,et al.  The qualitative content analysis process. , 2008, Journal of advanced nursing.

[12]  R. Collins.  The Credential Society: An Historical Sociology of Education and Stratification , 1979 .

[13]  Rinze Benedictus,et al.  Fewer numbers, better science , 2016, Nature.

[14]  P. Gross,et al.  COLLEGE LIBRARIES AND CHEMICAL EDUCATION. , 1927, Science.

[15]  Henk F. Moed,et al.  Handbook of Quantitative Science and Technology Research: The Use of Publication and Patent Statistics in Studies of S&T Systems , 2004 .

[16]  Andrew Abbott,et al.  The Problem of Excess , 2014 .

[17]  L. Butler,et al.  Assessing university research: A plea for a balanced approach , 2007 .

[18]  J. Nicolaisen.  Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics , 2010 .

[19]  R. Whitley.  The Intellectual and Social Organization of the Sciences (Second Edition: with new introductory chapter entitled 'Science Transformed? The Changing Nature of Knowledge Production at the End of the Twentieth Century') , 1984 .

[20]  Gissur Ó Erlingsson,et al.  Att möta den högre utbildningens utmaningar , 2016 .

[21]  Ludo Waltman,et al.  The inconsistency of the h-index , 2011, J. Assoc. Inf. Sci. Technol..

[22]  Gemma Derrick,et al.  "A number you just can't get away from": Characteristics of Adoption and the Social Construction of Metric Use by Researchers , 2013 .

[23]  Rangnar Nilsson.  God vetenskap - hur forskares vetenskapsuppfattningar uttryckta i sakkunningutlåtanden förändras i tre skilda discipliner , 2009 .

[24]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[25]  H. Schuman,et al.  Citation counts and social comparisons: Scientists' use and evaluation of citation index data , 1990 .

[26]  D. Hicks.  The Four Literatures of Social Science , 2004 .

[27]  Kayvan Kousha,et al.  The Metric Tide: Literature Review , 2015 .

[28]  Bruce Macfarlane,et al.  Defining and Rewarding Academic Citizenship: The implications for university promotions policy , 2007 .

[29]  Marion Fourcade,et al.  The Superiority of Economists , 2015 .

[30]  Eugene Garfield,et al.  New factors in the evaluation of scientific literature through citation indexing , 1963 .

[31]  Loet Leydesdorff,et al.  Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report , 2016, Scientometrics.

[32]  Vincent Larivière,et al.  History of the journal impact factor: Contingencies and consequences , 2009, Scientometrics.

[33]  Jochen Gläser,et al.  Advantages and dangers of ‘remote’ peer evaluation , 2005 .

[34]  Michael Graber,et al.  Publish or Perish? The Increasing Importance of Publications for Prospective Economics Professors in Austria, Germany and Switzerland , 2008 .

[35]  Paul Wouters,et al.  Evaluation practices and effects of indicator use : a literature review , 2016 .

[36]  B. Malsch,et al.  Journal ranking effects on junior academics: Identity fragmentation and politicization , 2015 .

[37]  Sarah de Rijcke,et al.  Quantified academic selves: the gamification of research through social networking services , 2016, Inf. Res..

[38]  Steve Woolgar,et al.  Beyond the citation debate: towards a sociology of measurement technologies and their use in science policy , 1991 .

[39]  Anthony F. J. van Raan,et al.  Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises , 1996, Scientometrics.

[40]  Dennis Tourish,et al.  In defiance of folly: Journal rankings, mindless measures and the ABS guide , 2015 .

[41]  Alexander Rushforth,et al.  Accounting for Impact? The Journal Impact Factor and the Making of Biomedical Research in the Netherlands , 2015, Minerva.

[42]  Barry Bozeman,et al.  Peer Review and Evaluation of R&D Impacts , 1993 .

[43]  Birger Larsen,et al.  A review of the characteristics of 108 author-level bibliometric indicators , 2014, Scientometrics.

[44]  Jochen Gläser,et al.  The Changing Governance of the Sciences , 2007 .

[45]  Henk F. Moed,et al.  The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review , 2007 .

[46]  Álvaro Santana Acuña.  How Professors Think: Inside the Curious World of Academic Judgment , 2012 .

[47]  Gualberto Buela-Casal,et al.  What do the scientists think about the impact factor? , 2012, Scientometrics.

[48]  D. Aksnes,et al.  Researchers’ perceptions of citations , 2009 .