Proposing a taxonomy for performance measurement systems' failures

Failures of business performance measurement (BPM) systems are dispersedly discussed in the abundance of literature written. Due to the multi-disciplinarity of stakeholders and researchers involved, the basis of literature is expanding but not converging. The added value of this paper is twofold. Firstly, the nomenclature used in the BPM field is aligned and represented visually. Secondly, this paper compiles and discusses 36 identified failures of performance measurement systems (PMS), thereby proposing an easy taxonomy. The classification draws upon three layers: metric level, framework level and management level, with respectively 13, 9 and 14 failures. This paper holds information for both academics and business people. The former can employ the literature overview for further referencing and can use it as a guideline to construct new BPM frameworks or systems, or adjust old ones. By reading this paper, people from the field create an awareness of risks involved when implementing a PMS. Alternatively, they can use it as a checklist in their current situation or tool for easy communication. Further research is necessary, both for tackling the problems listed and for looking into the correlation of the presented failures.

[1]  Yakov Ben-Haim,et al.  Uncertainty, probability and information-gaps , 2004, Reliab. Eng. Syst. Saf..

[2]  Steve Mason,et al.  Towards a definition of a business performance measurement system , 2007 .

[3]  Kevin Crowston,et al.  Methods for modeling and supporting innovation processes in SMEs , 2005 .

[4]  F. Johne,et al.  Success Factors in Product Innovation: A Selective Review of the Literature , 1988 .

[5]  Bob Kali,et al.  Strategic Performance Measurement , 2022 .

[6]  Hanne Nørreklit,et al.  The American balanced scorecard versus the French tableau de bord: the ideological dimension , 2004 .

[7]  Gregory G. Dess,et al.  Measuring organizational performance in the absence of objective measures: The case of the privately-held firm and conglomerate business unit , 1984 .

[8]  Christian Stummer,et al.  Research and development project selection and resource allocation: a review of quantitative modelling approaches , 1999 .

[9]  Mh Monique Jansen-Vullers,et al.  Business process simulation - tool survey , 2006 .

[10]  Andrew Pettigrew,et al.  Competitiveness and the Management of Strategic Change Processes: A Research Agenda , 1989 .

[11]  M. Smith,et al.  Implementing strategically aligned performance measurement in small firms , 2007 .

[12]  Andrea Rangone,et al.  An analytical hierarchy process framework for comparing the overall performance of manufacturing departments , 1996 .

[13]  Dorin Andreescu,et al.  Measuring Business Performance , 2010 .

[14]  Andy Neely,et al.  Finding performance: the new discipline in management , 2002 .

[15]  G. Musser.  Flaw of Averages , 2005 .

[16]  Juite Wang,et al.  A fuzzy set approach for R&D portfolio selection using a real options valuation model , 2007 .

[17]  Chrysanthos Dellarocas,et al.  Harnessing Crowds: Mapping the Genome of Collective Intelligence , 2009 .

[18]  Arthur M. Schneiderman,et al.  WHY BALANCED SCORECARDS FAIL , 2013 .

[19]  Karl T. Ulrich,et al.  Special Issue on Design and Development: Product Development Decisions: A Review of the Literature , 2001, Manag. Sci..

[20]  Andy Neely,et al.  The Performance Prism: The Scorecard for Measuring and Managing Business Success , 2002 .

[21]  Andy Neely,et al.  Designing, implementing and updating performance measurement systems , 2000 .

[22]  A. Neely.  The evolution of performance measurement research: Developments in the last decade and a research agenda for the next , 2005 .

[23]  A. Neely.  The performance measurement revolution: why now and what next? , 1999 .

[24]  Ronald N. Kostoff.  Metrics for planning and evaluating science and technology , 1998 .

[25]  D. Mccloskey,et al.  The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives , 2008 .

[26]  Douglas W. Hubbard,et al.  How to Measure Anything: Finding the Value of "Intangibles" in Business , 2007 .

[27]  Umit S. Bititci,et al.  Dynamics of performance measurement systems , 2000, APMS.

[28]  A. Jacoby.  Performance in the front-end of innovation : linking strategy to requirements , 2012 .

[29]  Luca Mari,et al.  A model for R&D performance measurement , 2011 .

[30]  Cornelius Herstatt,et al.  Approaches to the "fuzzy front end" of innovation , 1999 .

[31]  Anthony S. White,et al.  A Performance Measurement Paradigm for Integrating Strategy Formulation: A Review of Systems and Frameworks , 2005 .

[32]  Sai S. Nudurupati,et al.  State of the art literature review on performance measurement , 2011, Comput. Ind. Eng..

[33]  Kelvin F. Cross,et al.  Measure Up!: Yardsticks for Continuous Improvement , 1991 .

[34]  Geoff Lockett,et al.  Modeling Strategic Decision Making and Performance Measurements at ICI Pharmaceuticals , 1991 .

[35]  David Sinclair,et al.  Performance measurement: a critical analysis of the literature with respect to total quality management , 2000 .

[36]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[37]  Fredrik Nilsson,et al.  Recent advances in performance management : the Nordic case , 2002 .

[38]  A. Griffin,et al.  An Interim Report on Measuring Product Development Success and Failure , 1993 .

[39]  Efraim Turban,et al.  Decision Support and Business Intelligence Systems (8th Edition) , 2006 .

[40]  Douglas M. Lambert,et al.  Supply Chain Management: Processes, Partnerships, Performance , 2004 .

[41]  N. Geeraert,et al.  Measures Matter , 2014 .

[42]  R. Eccles.  The performance measurement manifesto. , 1991, Harvard business review.

[43]  G. Leeuwen.  Innovation and performance: A collection of microdata studies , 2009 .

[44]  Kelvin F. Cross,et al.  The “SMART” way to define and sustain success , 1988 .

[45]  Andy Neely,et al.  The success and failure of performance measurement initiatives: Perceptions of participating managers , 2002 .

[46]  V. Ridgway.  Dysfunctional Consequences of Performance Measurements , 1956 .

[47]  Akyay Uygur,et al.  EFQM Excellence Model , 2013 .

[48]  William E. Souder,et al.  Towards a Scale for Measuring Technology in New Product Innovations : Research Policy , 1985 .

[49]  G. Stewart.  The Quest for Value: A Guide for Senior Managers , 1991 .

[50]  S. Bird,et al.  Performance indicators: good, bad, and ugly , 2004 .

[51]  Ronald N. Kostoff,et al.  The metrics of science and technology , 2001, Scientometrics.

[52]  Noah P. Barsky,et al.  Utilizing the balanced scorecard for R&D performance measurement , 2004 .

[53]  Laurence Turcksin,et al.  Multi actor multi criteria analysis (MAMCA) as a tool to support sustainable decisions: State of use , 2012, Decis. Support Syst..

[54]  Umit Bititci,et al.  Towards a contingency approach to performance measurement: an empirical study in Scottish SMEs , 2007 .

[55]  Vittorio Chiesa,et al.  Searching for an effective measure of R&D performance , 1996 .

[56]  Kulwant S. Pawar,et al.  Performance measurement for product design and development in a manufacturing environment , 1999 .

[57]  Marc H. Meyer,et al.  Metrics for Managing Research and Development in the Context of the Product Family , 1997 .

[58]  赤尾 洋二,et al.  QFD : the customer-driven approach to quality planning and deployment , 1994 .

[59]  Maria Goddard,et al.  Performance management and Operational Research: a marriage made in heaven? , 2002, J. Oper. Res. Soc..

[60]  Alan W. Pearson,et al.  Performance measurement in industrial R&D , 2000 .

[61]  Jacky Holloway,et al.  Investigating the impact of performance measurement , 2001 .

[62]  John R. Hauser,et al.  Metrics: you are what you measure! , 1998 .

[63]  P. Trott.  Innovation Management And New Product Development , 2004 .

[64]  Søllerød Park,et al.  Managing for Results: , 1963, How Ten Global Cities Take On Homelessness.

[65]  Richard R. Reilly,et al.  Understanding Antecedents of New Product Development Speed: A Meta-Analysis , 2010 .

[66]  Albert H. Rubenstein.  50 years of engineering and technology management , 2004, IEEE Transactions on Engineering Management.

[67]  Susan Carlson Skalak.  House of Quality , 2002 .

[68]  Jeffrey D. Ford,et al.  Conceptual Issues of Linkage in the Assessment of Organizational Performance , 1982 .

[69]  Ronald N. Kostoff,et al.  The unintended consequences of metrics in technology evaluation , 2007, J. Informetrics.

[70]  Vittorio Chiesa,et al.  Evaluating Intangible Assets: The Measurement of R&D Performance , 2001 .

[71]  P. Garengo,et al.  Performance Measurement Systems in SMEs: A Review for a Research Agenda , 2005 .

[72]  Eliezer Geisler,et al.  The metrics of technology evaluation: where we stand and where we should go from here , 2002, Int. J. Technol. Manag..

[73]  Umit Bititci,et al.  Quantitative models for performance measurement system , 2000 .

[74]  Antonio Davila,et al.  Performance Measurement and Control Systems for Implementing Strategy: Text and Cases , 1999 .

[75]  Raili Pollanen,et al.  Performance measurement in municipalities: Empirical evidence in Canadian context , 2005 .

[76]  Umit Bititci,et al.  Integrated performance measurement systems: a development guide , 1997 .

[77]  T. Davenport.  Competing on analytics. , 2006, Harvard business review.

[78]  Anders Wall,et al.  Evaluation of performance in a product development context , 2010 .

[79]  K. Debackere,et al.  Measuring Progress and Evolution in Science and Technology - Ii: The Multiple Uses of Technometric Indicators , 2002 .

[80]  C. Bloch,et al.  Oslo Manual - Guidelines for Collecting and Interpreting Innovation Data, 3rd edition: Proposed Guidelines for Collecting and Interpreting Innovation Data , 2005 .

[81]  J. Manzoni,et al.  Strategic Performance Measurement: Benefits, Limitations and Paradoxes , 2010 .

[82]  Umit Bititci,et al.  Strategy Management through Quantitative Modelling of Performance Measurement Systems , 2001 .

[83]  Andy Neely,et al.  Performance measurement system design: developing and testing a process‐based approach , 2000 .

[84]  Thomas H. Davenport,et al.  Automated Decision Making Comes of Age , 2005 .

[85]  R. Westbrook,et al.  Linking Actions to Profits in Strategic Decision Making , 2001 .

[86]  Keith D. Brouthers,et al.  Driving blind: Strategic decisionmaking in small companies , 1998 .

[87]  P. Coughlan,et al.  Development of a technical innovation audit , 1996 .

[88]  R. Jacobs,et al.  Alternative Methods to Examine Hospital Efficiency: Data Envelopment Analysis and Stochastic Frontier Analysis , 2001, Health care management science.

[89]  Michel Lebas,et al.  A conceptual and operational delineation of performance , 2002 .

[90]  D. Otley.  Business performance measurement Theory and practice , 2003 .

[91]  Prakash J. Singh,et al.  Research in the Innovation Management Area: Lessons fromQuality Management , 2006 .

[92]  B Aurélia,et al.  Innovation management and new product development , 2013 .

[93]  Peter M. Senge,et al.  From theory to practice: research territory, processes and structure at an organizational learning centre , 1996 .

[94]  R. Kaplan,et al.  Measure Costs Right: Make the Right Decisions , 1988 .

[95]  Jacky Holloway,et al.  What Have We Learned? Themes from the Literature on Best-Practice Benchmarking , 2007 .

[96]  Andy Neely,et al.  Getting the measure of your business , 2002 .

[97]  Harlow B. Cohen.  The performance paradox , 1998 .

[98]  Andy Neely,et al.  A framework of the factors affecting the evolution of performance measurement systems , 2002 .

[99]  Klas Eric Soderquist,et al.  Use and impact of performance measurement results in R&D and NPD: an exploratory study , 2004 .

[100]  Andy Neely,et al.  Performance measurement frameworks: A review , 2007 .