Design principles for research & development performance measurement systems: a systematic literature review

Goal: This paper aims to set the foundation on which a performance measurement system for the R&D function will be developed, assuming it not only as an organizational division, but also as a comprehensive process in order to better evaluate its efficiency and effectiveness. Design/Methodology/Approach: This paper adopts a Design Science Research strategy as its methodological basis. It focuses on bringing guidelines from the body of organizational theory of knowledge to design effective management solutions for an R&D performance measurement system. Results: A set of well-defined Design Principles to guide performance measurement system designs for R&D management. Limitations of the investigation: Even though the design principles have been systematically obtained from the literature review, there was no adequacy test of these principles in practical cases, which should be conducted in future research. Practical Implications: Managers and leaders of R&D teams may use theory-grounded guidelines to design their performance systems and to adjust such systems to their specific needs. Originality/ Value: The main practical contribution of the study findings is to provide a comprehensive set of guidelines to design a R&D Performance Measurement System, rather than proposing another new complete theoretical framework to add to the literature.

[1]  Paul A. Schumann,et al.  Measuring R&D Performance , 1995 .

[2]  Joakim Björkdahl,et al.  Assessing firm capabilities for innovation , 2012 .

[3]  Gina Colarelli O'Connor,et al.  "Organizing for Breakthrough Innovation: Leadership, Structures, and Processes" , 2014 .

[4]  Vittorio Chiesa,et al.  Evaluating Intangible Assets: The Measurement of R&D Performance , 2001 .

[5]  A. Johansson,et al.  Innovation in Service Organizations : The development of a suitable innovation measurement system , 2015 .

[6]  Jafar Rezaei,et al.  Evaluating firms' R&D performance using best worst method. , 2018, Evaluation and program planning.

[7]  Daniel Pacheco Lacerda,et al.  Design Science Research: A Method for Science and Technology Advancement , 2014 .

[8]  Federico Frattini,et al.  TOWARDS A SYSTEM OF PERFORMANCE MEASURES FOR RESEARCH ACTIVITIES: NIKEM RESEARCH CASE STUDY , 2006 .

[9]  O. Gassmann,et al.  Managing R&D and New Product Development , 2014 .

[10]  Fiorenzo Franceschini,et al.  Management by Measurement: Designing Key Indicators and Performance Measurement Systems , 2007 .

[11]  Dirk Lüttgens,et al.  Measuring open innovation : A toolkit for successful innovation teams , 2014, Performance 2014.

[12]  Robert Szakonyi,et al.  Measuring R&D Effectiveness—I , 1994 .

[13]  Ville Ojanen,et al.  Coping with the multiple dimensions of R&D performance analysis , 2006, Int. J. Technol. Manag..

[14]  A. Georges L. Romme,et al.  Special Issue: Organizational Design: Construction Principles and Design Rules in the Case of Circular Design , 2006, Organ. Sci..

[15]  Jan Bilderbeek,et al.  R&D Performance Measurement: More than Choosing a Set of Metrics , 1997 .

[16]  P. Coughlan,et al.  Development of a technical innovation audit , 1996 .

[17]  K. H. Ulltveit-Moe,et al.  R&D, International Sourcing, and the Joint Impact on Firm Performance , 2015 .

[18]  Sofia Ritzén,et al.  Designing and implementing a method to build innovation capability in product development teams , 2013 .

[19]  Eva Mulero-Mendigorri,et al.  Content Validation of a Measure of R&D Effectiveness , 2005 .

[20]  Mario Bunge,et al.  Scientific Research II , 1967 .

[21]  Gretchen B. Jordan,et al.  Assessing and improving the effectiveness of National Research Laboratories , 2003, IEEE Trans. Engineering Management.

[22]  Claudio Cruz-Cázares,et al.  You can’t manage right what you can’t measure well: Technological innovation efficiency☆ , 2013 .

[23]  John Bessant,et al.  Managing Innovation: Integrating Technological, Market, and Organizational Change, 2nd Edition , 2001 .

[24]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[25]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[26]  Tina Skerlj.  Measuring Innovation Excellence: Measurement Framework for PWC’s Wheel of Innovation Excellence Concept , 2014 .

[27]  Andy Neely,et al.  Performance measurement system design: developing and testing a process‐based approach , 2000 .

[28]  R. Rothwell.  Towards the Fifth‐generation Innovation Process , 1994 .

[29]  Rene Cordero,et al.  The measurement of innovation performance in the firm: An overview , 1990 .

[30]  Alan W. Pearson,et al.  Performance measurement in industrial R&D , 2000 .

[31]  David Parmenter,et al.  Key Performance Indicators: Developing, Implementing,and Using Winning KPIs , 2007 .

[32]  Andrew Cooke,et al.  Design principles for the development of measurement systems for research and development processes , 1997 .

[33]  M. M. Carvalho,et al.  INNOVATION MANAGEMENT: A SYSTEMATIC LITERATURE ANALYSIS OF THE INNOVATION MANAGEMENT EVOLUTION , 2016 .

[34]  Will Kaydos.  Measuring, managing, and maximizing performance : what every manager needs to know about quality and productivity to make real improvements in performance , 1991 .

[35]  H. Chesbrough.  The Era of Open Innovation , 2003 .

[36]  Ernest R. Gilmont.  Making Innovation Work: How to Manage It, Measure It, and Profit from It , 2007 .

[37]  Sofia Ritzén,et al.  Re-thinking innovation measurement to manage innovation-related dichotomies in practice , 2012 .

[38]  S. Ritzén,et al.  Exploring the Use of Innovation Performance Measurement to Build Innovation Capability in a Medical Device Company , 2014 .

[39]  Minna Saunila,et al.  A conceptual framework for the measurement of innovation capability and its effects , 2012 .

[40]  Vittorio Chiesa,et al.  How do measurement objectives influence the R&D performance measurement system design?: Evidence from a multiple case study , 2007 .

[41]  Manish Godse,et al.  Towards a holistic enterprise innovation performance measurement system , 2014 .

[42]  E. C. Galloway.  Evaluating R & D Performance — Keep It Simple , 1971 .

[43]  Donald S. Remer,et al.  Measuring R&D Productivity , 1985 .

[44]  M. Sawhney,et al.  The 12 different ways for companies to innovate , 2012, IEEE Engineering Management Review.

[45]  Mario Sergio Salerno,et al.  Reconfigurable organisation to cope with unpredictable goals , 2009 .

[46]  Nauman Bin Ali,et al.  Towards innovation measurement in the software industry , 2013, J. Syst. Softw..

[47]  Andre Benaim,et al.  Innovation Capabilities – Measurement, Assessment and Development , 2015 .

[48]  Morten T. Hansen,et al.  The innovation value chain. , 2007, Harvard business review.

[49]  A. Fusfeld,et al.  Assessing the Value of Your Technology , 1995 .

[50]  A. Verhaeghe,et al.  Managing Innovation in a Knowledge Intensive Technology Organisation (Kito) , 2002 .

[51]  Minna Saunila,et al.  Performance management through innovation capability in SMEs , 2014 .

[52]  J. Tidd,et al.  Innovation Management Practices: Cross‐Sectorial Adoption, Variation, and Effectiveness , 2016 .

[53]  A. Muller,et al.  Metrics for innovations: Guidelines for developing a customized suite of innovation metrics , 2005 .

[54]  K. Goffin,et al.  Innovation Management: Strategy and Implementation using the Pentathlon Framework , 2005 .

[55]  Noah P. Barsky,et al.  Utilizing the balanced scorecard for R&D performance measurement , 2004 .

[56]  Michael E. McGrath,et al.  The R&D effectiveness index: A metric for product development performance , 1994 .

[57]  John R. Hauser,et al.  Metrics to evaluate R,D&E , 1997 .

[58]  Edson Pinheiro de Lima,et al.  Framing Operations and Performance Strategic Management System Design Process , 2008 .

[59]  Adriana de Paula Lacerda Santos,et al.  INDUSTRY 4.0: glitter or gold? A systematic review , 2018, Brazilian Journal of Operations & Production Management.

[60]  Lawrence H. Linden,et al.  Improving the Return on R&D—II , 1985 .

[61]  Joan Ernst van Aken,et al.  Design Science: Valid Knowledge for Socio-technical System Design , 2012 .

[62]  François Vernadat,et al.  Enterprise modeling and integration : principles and applications , 1996 .

[63]  Vittorio Chiesa,et al.  Designing a performance measurement system for the research activities: A reference framework and an empirical study , 2008 .

[64]  Raynold A. Svenson,et al.  Measuring R&D Productivity , 1988 .

[65]  M. Moser.  Measuring Performance in R&D Settings , 1985 .

[66]  A. Gawer,et al.  Industry Platforms and Ecosystem Innovation , 2013 .

[67]  A. Griffin,et al.  An Interim Report on Measuring Product Development Success and Failure , 1993 .

[68]  Elizabeth Webster,et al.  Examining Biases in Measures of Firm Innovation , 2004 .

[69]  J. Bessant,et al.  Innovation Management Measurement: A Review , 2006 .

[70]  William E. Souder,et al.  Measuring R&D Performance-State of the Art , 1997 .

[71]  A. Griffin,et al.  PDMA Success Measurement Project: Recommended Measures for Product Development Success and Failure , 1996 .

[72]  P. Hertog,et al.  Capabilities for managing service innovation: towards a conceptual framework , 2010 .

[73]  Vittorio Chiesa,et al.  Evaluation and Performance Measurement of Research and Development , 2009 .

[74]  J. Aken.  Management Research Based on the Paradigm of the Design Sciences: The Quest for Field-Tested and Grounded Technological Rules , 2004 .

[75]  A. Neely.  The performance measurement revolution: why now and what next? , 1999 .

[76]  Luca Mari,et al.  A model for R&D performance measurement , 2011 .