EVALUATIONS: HIDDEN COSTS, QUESTIONABLE BENEFITS, AND SUPERIOR ALTERNATIVES

Research evaluation is praised as the symbol of modern quality management. We claim firstly, performance evaluations in research have higher costs than normally assumed, because the evaluated persons and institutions systematically change their behavior and develop counter strategies. Moreover, intrinsic work motivation is crowded out and undesired lock-in effects take place. Secondly, the benefits of performance evaluations are questionable. Evaluations provide too little information relevant for decision-making. In addition, they lose importance due to new forms of scientific cooperation on the internet. Thirdly, there exist superior alternatives. They consist in careful selection and supportive process coaching i?½ and then leave individuals and research institutions to direct themselves.

[1]  B. Kogut,et al.  What Firms Do? Coordination, Identity, and Learning , 1996 .

[2]  S. Kerr.  On the folly of rewarding A, while hoping for B. , 1975 .

[3]  Markus Gmür,et al.  Co-citation analysis and the search for invisible colleges: A methodological evaluation , 2004, Scientometrics.

[4]  B. Nooteboom.  Learning by Interaction: Absorptive Capacity, Cognitive Distance and Governance , 2000 .

[5]  E. Fehr,et al.  Do Incentive Contracts Crowd Out Voluntary Cooperation? , 2000 .

[6]  Richard Smith,et al.  Journal accused of manipulating impact factor , 1997 .

[7]  J. Cyr.  Formal control and trustworthiness: shall the twain never meet? , 2008 .

[8]  W D Pierce,et al.  Pervasive negative effects of rewards on intrinsic motivation: The myth continues , 2001, The Behavior analyst.

[9]  Remo Guidieri.  Res , 1995, RES: Anthropology and Aesthetics.

[10]  Jürgen Renn,et al.  Challenges from the past. Innovative structures for science and the contribution of the history of science , 2002 .

[11]  B. Frey,et al.  Publishing as Prostitution? Choosing between One's Own Ideas and Academic Failure , 2002 .

[12]  Eric W. Ford,et al.  Community of scholars: An exploratory study of management laureates , 2002, Scientometrics.

[13]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[14]  L. Butler,et al.  Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts , 2003 .

[15]  Kim B. Clark,et al.  The power of modularity , 2000 .

[16]  Kim B. Clark,et al.  Design Rules: The Power of Modularity , 2000 .

[17]  E. H. Kim.  Are Elite Universities Losing Their Competitive Edge ? , 2006 .

[18]  Michael Power,et al.  The audit society : rituals of verification , 1997 .

[19]  小山 和伸.  Academy of Management Review : 抄録雑誌の概要 , 1987 .

[20]  A. Kieser.  Going Dutch? Was lehren niederländische Erfahrungen mit der Evaluation universitärer Forschung? , 1998 .

[21]  Bruno S. Frey,et al.  Successful management by motivation : balancing intrinsic and extrinsic incentives , 2002 .

[22]  Bruno S. Frey,et al.  Yes, Managers Should Be Paid Like Bureaucrats , 2005, SSRN Electronic Journal.

[23]  Adam Brand,et al.  Knowledge Management and Innovation at 3M , 1998, J. Knowl. Manag..

[24]  Eugene Garfield,et al.  Editors are justified in asking authors to cite equivalent references from same journal , 1997 .

[25]  Michael Muller-Camen,et al.  Changes in Academic Work and the Chair Regime: The Case of German Business Administration Academics , 2005 .

[26]  M. Biagioli.  From Book Censorship to Academic Peer Review , 2002 .

[27]  Stephen M. Ross,et al.  Authority and Commitment : Why Universities , Like Legislatures , Are Not Organized as Firms , 2006 .

[28]  Michelle Gittelman,et al.  Does Good Science Lead to Valuable Knowledge? Biotechnology Firms and the Evolutionary Logic of Citation Patterns , 2003, Manag. Sci..

[29]  Edwin Klijn,et al.  Royal Netherlands Academy of Arts and Sciences , 2000 .

[30]  Paul Mizen,et al.  Goodhart’s Law: Its Origins, Meaning and Implications for Monetary Policy , 2001 .

[31]  Donald Gillies,et al.  Hempelian and Kuhnian approaches in the philosophy of medicine: the Semmelweis case. , 2005, Studies in history and philosophy of biological and biomedical sciences.

[32]  Arie W. Kruglanski,et al.  Covariation, discounting, and augmentation: Towards a clarification of attributional principles , 1978 .

[33]  Kathleen M. Eisenhardt,et al.  Control: Organizational and Economic Approaches , 1985 .

[34]  Randall G. Holcombe,et al.  The National Research Council Ranking of Research Universities: Its Impact on Research in Economics , 2004 .

[35]  R. Gibbons.  Incentives in Organizations , 1998 .

[36]  Alfred Kieser,et al.  Success Factor Research , 2005 .

[37]  B. Frey,et al.  Motivation crowding theory , 2001 .

[38]  Jerry G. Thursby.  What Do We Say about Ourselves and What Does It Mean? Yet Another Look at Economics Department Research , 2000 .

[39]  W. Ouchi.  The Relationship Between Organizational Structure and Organizational Control. , 1977 .

[40]  William H. Starbuck,et al.  How Much Better are the Most Prestigious Journals? The Statistics of Academic Publication , 2005, Organ. Sci..

[41]  T. Tyler,et al.  The Social Psychology of Procedural Justice , 1988 .

[42]  Bruno S. Frey,et al.  Shareholders Should Welcome Knowledge Workers as Directors , 2006 .

[43]  M. V. Simkin,et al.  Copied citations create renowned papers , 2003, cond-mat/0305150.

[44]  김재영.  Siting Hazardous Waste Treatment Facilities : The NIMBY Syndrome , 1992 .

[45]  A. Falk,et al.  The hidden costs of control , 2006 .

[46]  Jared D. Harris,et al.  Incentives to Cheat: The Influence of Executive Compensation and Firm Performance on Financial Misrepresentation , 2006, Organ. Sci..

[47]  Melissa S. Anderson,et al.  Scientists behaving badly , 2005, Nature.

[48]  M. Power.  The Audit Explosion , 1994 .

[49]  Donald J. Gillies,et al.  Lessons from the History and Philosophy of Science regarding the Research Assessment Exercise , 2007, Royal Institute of Philosophy Supplement.

[50]  Siegwart Lindenberg.  Intrinsic motivation in a new light , 2001 .

[51]  K. Popper,et al.  Conjectures and refutations;: The growth of scientific knowledge , 1972 .

[52]  S. Fletcher.  Guardians of Science: Fairness and Reliability of Peer Review , 1994 .

[53]  B. Frey,et al.  Motivation, Knowledge Transfer, and Organizational Forms , 2000 .

[54]  J. Gans,et al.  How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists , 1994 .

[55]  Paul R. Milgrom,et al.  Multitask Principal–Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design , 1991 .

[56]  Linda Argote,et al.  Managing Knowledge in Organizations: An Integrative Framework and Review of Emerging Themes , 2003, Manag. Sci..

[57]  John Gennard,et al.  The state of the field in UK management research: reflections of the research assessment exercise (rae) panel , 2003 .

[58]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 1 , 1998 .

[59]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 2 , 1998 .

[60]  Peter Weingart,et al.  Impact of bibliometrics upon the science system: Inadvertent consequences? , 2005, Scientometrics.

[61]  F. Mehring,et al.  MLA millennial questionnaire on university issues - Reply , 2000 .

[62]  B. Frey,et al.  Motivation, Knowledge Transfer, and Organizational Forms , 1999 .

[63]  Clayton M. Christensen,et al.  CUSTOMER POWER, STRATEGIC INVESTMENT, AND THE FAILURE OF LEADING FIRMS , 1996 .

[64]  Kevin G. Corley,et al.  Being Good Versus Looking Good: Business School Rankings and the Circean Transformation From Substance to Image , 2002 .

[65]  Teresa M. Amabile,et al.  How to kill creativity. , 1998, Harvard business review.

[66]  B. Frey.  Not Just for the Money: An Economic Theory of Personal Motivation , 1998 .

[67]  Howard Kunreuther,et al.  The Dilemma of Siting a High-Level Nuclear Waste Repository , 1995 .

[68]  Y. Pasadeos,et al.  Organizational Analysis in North America and Europe: A Comparison of Co-citation Networks , 1995 .

[69]  Scott E. Masten,et al.  Authority and Commitment: Why Universities, Like Legislatures, are Not Organized as Firms , 2005 .

[70]  Tilman Brück,et al.  Do Eurozone Countries Cheat with Their Budget Deficit Forecasts? , 2006 .

[71]  William H. Starbuck,et al.  Methodological Challenges Posed by Measures of Performance , 2004 .

[72]  Hans-Dieter Daniel,et al.  Publications as a measure of scientific advancement and of scientists' productivity , 2005, Learn. Publ..

[73]  Margit Osterloh,et al.  Human Resources Management and Knowledge Creation , 2005 .

[74]  Canice Prendergast.  The Provision of Incentives in Firms , 1999 .

[75]  Arthur G. Bedeian,et al.  The Manuscript Review Process , 2003 .

[76]  Kevin G. Corley,et al.  The Rankings Game: Managing Business School Reputation , 2000 .

[77]  Iris Bohnet,et al.  Fairness and competence in democratic decisions , 1997 .

[78]  Arthur G. Bedeian,et al.  Peer Review and the Social Construction of Knowledge in the Management Discipline , 2004 .

[79]  Hugh Willmott,et al.  Just How Managed is the McUniversity? Craig Prichard, Hugh Willmott , 1997 .

[80]  Lucian Arye Bebchuk,et al.  Pay Without Performance: Overview of the Issues , 2005 .

[81]  K. Popper,et al.  Conjectures and refutations;: The growth of scientific knowledge , 1972 .

[82]  A. Rustichini,et al.  Pay Enough or Don't Pay at All , 2000 .

[83]  Ernst Fehr,et al.  Fairness and Incentives in a Multi-Task Principal-Agent Model , 2004 .

[84]  Margit Osterloh,et al.  Open Source Software Development - Just Another Case of Collective Invention? , 2004 .

[85]  Thomas D. Cook,et al.  Metaevaluation Research , 1978 .

[86]  Robert K. Toutkoushian,et al.  Using Publications Counts to Measure an Institution's Research Productivity , 2003 .

[87]  Jan Venselaar,et al.  DESIGN RULES , 1999 .

[88]  Andy Neely,et al.  Business performance measurement: Performance measurement – practical applications , 2002 .

[89]  Celia Moore,et al.  New Developments in Performance Management , 2005 .

[90]  Ilia D. Dichev.  How Good Are Business School Rankings , 1999 .

[91]  B. Rabe,et al.  Alternatives to NIMBY gridlock: voluntary approaches to radioactive waste facility siting in Canada and the United States , 1994 .

[92]  Teresa M. Amabile,et al.  Creativity In Context: Update To The Social Psychology Of Creativity , 1996 .

[93]  R. Langlois.  Modularity in technology and organization , 2002 .

[94]  Paul Mizen,et al.  Central Banking, Monetary Theory and Practice , 2003 .

[95]  Peter R. Wellhöfer,et al.  Evaluation der Evaluation , 2002 .

[96]  Paul R. Milgrom,et al.  Economics, Organization and Management , 1992 .

[97]  Martin Parker,et al.  The McUniversity: Organization, Management and Academic Subjectivity , 1995 .

[98]  Mona V. Makhija,et al.  The Role Of Organizational Controls In Managing Knowledge , 2006 .

[99]  J. Tirole,et al.  Intrinsic and Extrinsic Motivation , 2003 .

[100]  D. Cicchetti.  The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation , 1991, Behavioral and Brain Sciences.

[101]  Laura B. Cardinal.  Technological Innovation in the Pharmaceutical Industry: The Use of Organizational Control in Managing Research and Development , 2001 .

[102]  Kent E. Portney.  Siting Hazardous Waste Treatment Facilities: The NIMBY Syndrome, Kent E. Portney. 1991. Auburn House, Westport, CT. 200 pages. ISBN: 0-86569-016-2. $37.95 , 1991 .

[103]  Armin Falk,et al.  Distrust - the Hidden Cost of Control , 2004, SSRN Electronic Journal.

[104]  David Adam,et al.  Citation analysis: The counting house , 2002, Nature.

[105]  W. Ouchi.  A Conceptual Framework for the Design of Organizational Control Mechanisms , 1979 .

[106]  Juan Miguel Campanario,et al.  UsingCitation Classics to study the incidence of serendipity in scientific discovery , 1996, Scientometrics.

[107]  L. Bebchuk,et al.  Pay without Performance, The Unfulfilled Promise of Executive Compensation, Part IV: Going Forward , 2004 .

[108]  Hans de Bruijn,et al.  Managing Performance in the Public Sector , 2002 .

[109]  E. Deci,et al.  A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. , 1999, Psychological bulletin.

[110]  George A. Akerlof.  An economic theorist's book of tales: The economics of caste and of the rat race and other woeful tales , 1976 .

[111]  S. Snell.  Control Theory In Strategic Human Resource Management: The Mediating Effect Of Administrative Information , 1992 .

[112]  Paul A. Argenti,et al.  Branding B-Schools: Reputation Management for MBA Programs , 2000 .

[113]  James D. Thompson.  Organizations in Action: Social Science Bases of Administrative Theory , 1967 .

[114]  Arthur G. Bedeian,et al.  The Gift of Professional Maturity , 2004 .

[115]  Antoinette Weibel,et al.  Formal Control and Trustworthiness , 2007 .

[116]  Petra Moog,et al.  Ökonomie der Evaluation von Schulen und Hochschulen. , 2004 .

[117]  Marshall H. Medoff,et al.  Evidence of a Harvard and Chicago Matthew Effect , 2006 .

[118]  E. Lawler,et al.  Information and control in organizations , 1978 .

[119]  Jack A. Nickerson,et al.  A Knowledge-based Theory of the Firm - A Problem-solving Perspective , 2004, Organ. Sci..

[120]  Nicolai J. Foss,et al.  ‘Tying the manager's hands’: constraining opportunistic managerial intervention , 2006 .

[121]  Manson.  Part 1: Research , 1982 .