Governance by Numbers. Does It Really Work in Research?

Abstract Performance evaluation in research is more and more based on numbers of publications, citations, and impact factors. In the wake of New Public Management output control has been introduced into research governance without taking into account the conditions necessary for this kind of control to work efficiently. It is argued that to evaluate research by output control is counterproductive. It induces to substitute the ‘taste for science’ by a ‘taste for publication’. Instead, input control by careful selection and socialization serves as an alternative.

[1]  B. Heintz.  Governance by Numbers. Zum Zusammenhang von Quantifizierung und Globalisierung am Beispiel der Hochschulpolitik , 2008 .

[2]  E. Deci,et al.  Self‐determination theory and work motivation , 2005 .

[3]  Edwin A. Locke,et al.  Has Goal Setting Gone Wild, or Have Its Attackers Abandoned Good Scholarship? , 2009 .

[4]  Teresa M. Amabile,et al.  Creativity In Context: Update To The Social Psychology Of Creativity , 1996 .

[5]  I. Janis.  Victims Of Groupthink , 1972 .

[6]  William H. Starbuck,et al.  How Much Better are the Most Prestigious Journals? The Statistics of Academic Publication , 2005, Organ. Sci..

[7]  D. Horrobin,et al.  Peer review of grant applications: a harbinger for mediocrity in clinical research? , 1996, The Lancet.

[8]  W. Espeland,et al.  Rankings and Reactivity: How Public Measures Recreate Social Worlds1 , 2007, American Journal of Sociology.

[9]  S. Kerr.  On the folly of rewarding A, while hoping for B. , 1975 .

[10]  Klaus M. Schmidt,et al.  Fairness and Incentives in a Multi-Task Principle-Agent Model , 2004 .

[11]  Harlow B. Cohen.  The performance paradox , 1998 .

[12]  A. Wandersman,et al.  Urban neighborhoods and mental health. Psychological contributions to understanding toxicity, resilience, and interventions. , 1998, The American psychologist.

[13]  Arthur G. Bedeian,et al.  The Manuscript Review Process , 2003 .

[14]  K. Merchant,et al.  Management Control Systems: Performance Measurement, Evaluation and Incentives , 2003 .

[15]  D. Simonton.  Creativity in Science: Chance, Logic, Genius, and Zeitgeist , 2004 .

[16]  B. Clark.  Creating entrepreneurial universities , 1998 .

[17]  L. Butler,et al.  Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts , 2003 .

[18]  J. Scott.  Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation , 1997 .

[19]  V. G. Ouchi,et al.  A conceptual framework for the design and organizational control mechanisms , 1979 .

[20]  Lutz Bornmann,et al.  The luck of the referee draw: the effect of exchanging reviews , 2009, Learn. Publ..

[21]  B. Frey,et al.  Motivation, Knowledge Transfer, and Organizational Forms , 2000 .

[22]  P. Rothwell,et al.  Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? , 2000, Brain : a journal of neurology.

[23]  Hugh Willmott,et al.  Just How Managed is the McUniversity? Craig Prichard, Hugh Willmott , 1997 .

[24]  Paul R. Milgrom,et al.  Multitask Principal–Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design , 1991 .

[25]  Henry L. Tosi,et al.  Effort, Performance, and Conscientiousness: An Agency Theory Perspective , 2007 .

[26]  H. Ursprung,et al.  Who is the ”Platz-Hirsch“ of the German Economics Profession? , 2007 .

[27]  P. David,et al.  Toward a new economics of science , 1994 .

[28]  Frank Donoghue.  The last professors , 2018 .

[29]  N. Adler,et al.  When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings , 2009 .

[30]  Teresa M. Amabile,et al.  Reality, intrinsic motivation, and creativity. , 1998 .

[31]  Marshall W. Meyer.  Can Performance Studies Create Actionable Knowledge if We Can’t Measure the Performance of the Firm? , 2005 .

[32]  B. Frey.  Not Just for the Money: An Economic Theory of Personal Motivation , 1998 .

[33]  R. Anthony,et al.  Management Control Systems , 2020, Management Control Systems and Tools for Internationalization Success.

[34]  S. Snell.  Control Theory In Strategic Human Resource Management: The Mediating Effect Of Administrative Information , 1992 .

[35]  ARMAMENT, DISARMAMENT AND NATIONAL SECURITY:: A Theoretical Duopoly Model of the Arms Race , 1979 .

[36]  M. V. Simkin,et al.  Copied citations create renowned papers , 2003, cond-mat/0305150.

[37]  James D. Thompson.  Organizations in Action: Social Science Bases of Administrative Theory , 1967 .

[38]  Teresa M. Amabile,et al.  Reward, Intrinsic Motivation, and Creativity , 1998 .

[39]  S. van Thiel,et al.  The Performance Paradox in the Public Sector , 2002 .

[40]  C. Donovan.  The qualitative future of research evaluation , 2007 .

[41]  Kathleen M. Eisenhardt,et al.  Control: Organizational and Economic Approaches , 1985 .

[42]  Daniel A. Levinthal,et al.  Hoping for A to Z While Rewarding Only A: Complex Organizations and Multiple Goals , 2009, Organ. Sci..

[43]  M. Schweitzer,et al.  Goals Gone Wild: The Systematic Side Effects of Overprescribing Goal Setting , 2009 .

[44]  Donald Gillies,et al.  Hempelian and Kuhnian approaches in the philosophy of medicine: the Semmelweis case. , 2005, Studies in history and philosophy of biological and biomedical sciences.

[45]  L. Kirsch.  The Management of Complex Tasks in Organizations: Controlling the Systems Development Process , 1996 .

[46]  J. Gans,et al.  How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists , 1994 .

[47]  Margit Osterloh,et al.  Governing Knowledge Work:: Transactional and Transformational Solutions , 2010 .

[48]  Stefan Thurner,et al.  Peer-review in a world with rational scientists: Toward selection of the average , 2010, 1008.4324.

[49]  D. Cicchetti.  The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation , 1991, Behavioral and Brain Sciences.

[50]  Bruno S. Frey,et al.  Tertium Datur: Pricing, Regulating and Intrinsic Motivation , 1992 .

[51]  Georg Schreyögg,et al.  Strategic Control: A New Perspective , 1987 .

[52]  Rakesh Khurana,et al.  Introduction to From Higher Aims to Hired Hands The Social Transformation of American Business Schools and the Unfulfilled Promise of Management as a Profession , 2010 .

[53]  M. Abernethy,et al.  Management control systems in research and development organizations: The role of accounting, behavior and personnel controls , 1997 .

[54]  Robert A. Simons.  Control in an Age of Empowerment , 2008 .

[55]  A. Oswald.  An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers , 2006, SSRN Electronic Journal.

[56]  Derek Bok,et al.  Universities in the Marketplace , 2003 .

[57]  F. Nijhuis,et al.  Prediction of scientific performance in clinical medicine , 1990 .

[58]  Mona V. Makhija,et al.  The Role Of Organizational Controls In Managing Knowledge , 2006 .

[59]  Robert K. Merton,et al.  Bureaucratic Structure and Personality , 1940 .

[60]  Juan Miguel Campanario,et al.  UsingCitation Classics to study the incidence of serendipity in scientific discovery , 1996, Scientometrics.

[61]  Amanda H. Goodall.  Highly Cited Leaders and the Performance of Research Universities. , 2008 .

[62]  B. Frey,et al.  Corporate Governance: What Can We Learn from Public Governance? , 2006 .

[63]  Giovanni Abramo,et al.  Allocative efficiency in public research funding: can bibliometrics help? , 2009, ArXiv.

[64]  Kevin G. Corley,et al.  Being Good Versus Looking Good: Business School Rankings and the Circean Transformation From Substance to Image , 2002 .

[65]  Peter Weingart,et al.  Impact of bibliometrics upon the science system: Inadvertent consequences? , 2005, Scientometrics.

[66]  Hugh Willmott,et al.  Commercialising Higher Education in the UK: The state, industry and peer review , 2003 .

[67]  A. Rustichini,et al.  Pay Enough or Don't Pay at All , 2000 .

[68]  M. Mahoney.  Publication prejudices: An experimental study of confirmatory bias in the peer review system , 1977, Cognitive Therapy and Research.

[69]  R. Posner.  From the new institutional economics to organization economics: with applications to corporate governance, government agencies, and legal institutions , 2010, Journal of Institutional Economics.